Building Secure Environments for CLI Code Agents

Speaker 1 [00:00]

Welcome to our session today, for this time, and I would like to introduce our speaker, Halad Nesbada, who will be speaking on building secure environments for the CLI code agents. Welcome to our speaker today.

Speaker 2 [00:29]

Thank you very much so We are going to discuss today. How we are going to build those environments to secure our container agents before a few things about me, I'm Harald I'm working at a company in Austria. It's called the next I'm there having the role of Python technical lead and in my day-to-day Activities I do lots of things with Python DevOps and also recently a lot with AI In my free time, I'm an open source maintainer, I do technical blogging, and I'm also a part-time lecturer at two universities in my area, and I also do photography. So before I start, I want to see some hands. I'm curious, how many of you are actually using CLI coding agents? Okay, perfect. And how many of you are actually running them directly on the machine without any sandboxing? Okay. Then who of you or how many of you are actually knowing what the agent is doing? Okay. So, what we want to do today, we want to look a bit at the risks of running those agents directly on your machine. We will look at a pattern I will propose, how you can isolate the agents so you can run them more securely and how to observe what they are doing, and then I will have a quick demo. I will show the system that I built, and I will also show you how you can look over the logs of the agent. So a bit of history, how this whole thing started. So back in 2017, we had a paper which introduced the whole transformer architecture. And with the transformer architecture, the first LLMs were built. And it took some time until 2022, which we had the main breakthrough of a chat interface that became very popular, and we still had this approach of you put some text in, you get some text out. A few years later, we had some new capabilities with those LLMs. The most important one is the tool use, because this one allowed now to introduce those agentic flows. So now models are capable of reading your files, also run commands, call external APIs. So this means we can now also act based on the information that the model generates. So it means we had an assistant mode, we had actually those applications that were able to explain things for us, they were not able to execute anything, so it means the risk for us was whatever we typed in the chat interface, that could be somehow sensitive or risky. Now we're moving at the agentic mode, where our tool is capable of executing commands, It can write and edit files and also make network requests. So this means we now have a much higher risk. How does the landscape now look for this agent? So from my experience, we have those three parts. We have the VS Code Forks, tools like Cursor and Windsurf. They are offering you the capability to use those agents. And we have the second iteration would be IDE plugins. You have things like Copilot or Juni or Augment Code. You can install them in your existing IDE and start using them. And more recently, we have the CLI agents. Things like Cloud Code, like Codex, or Gemini CLI, OpenCode. You can start them directly in your terminal and already connect them with your code base and start to do things. So this means those agents have access to your workspace. They can execute things. They have network access and so on. And the most important thing is the agent acts on your permissions. So it has actually the permissions the user has. So this means we are introducing some new risks, and this brings us to a concept that Simon Wilson introduced, which is called lethal trifecta, which means if you have private data and external access, and also your tool is acting based on untrusted content, you're really in danger. Because now, a lot of things can happen. Something like this scenario where we have accidental data exfiltration. This happens because you give the agent a specific prompt, the agent decides to help you based on some documentation. It reads that it needs to send the files to a specific API, and like this, your local environment file can't be now posted to some external vendor. Another scenario would be very messy clean-up, where you just say to the model, hey, I want to clean up all my files, and for some reason it reads not just your tests or temporary files that you created, but also your Git history. So probably you all heard about the recent attacks in the NPM package manager. This is also something that a model can do. It can read some sort of external documentation and decide this is the package that you need to solve the problem and will install it on your system and, I don't know, you may be seeing the Axios vulnerability where you suddenly had a crypto miner installed on your system just with an NPM package. Okay. Those were all hypothetical scenarios, but these are some real ones. So those are issues and stories where users used Cloud Code, and they suddenly got their home directory deleted. Which is not really that cool, and also the last story is a Mac user also was running Cloud Code with dangerous escape permissions, and also the whole home directory was wiped, which means the whole Mac was destroyed. OK. Also, I ran some queries on the repository of Cloud Code, and I see there are some issues. Also, the slides are shared afterwards. You can use the links, and you can also see it for yourself. So actually, what this means, you're kind of having a sort of demo class. So you are actually using the tool. You operate with it. It works fine until you have something catastrophic. So this leads me to the first isolation pattern I built. It's a project that's called Cloud Container. And the idea was to have an isolated runtime. In this case, put everything in a container and make sure that we limit the scope of what is actually accessible from the perspective of the agent. And in the meantime, I was also curious. I wanted to see what the agent is actually doing. So I ended up with this architecture. And my first iteration used this Anthropic base URL, which means I started Cloud Code, set the Anthropic base URL, which is actually meant for changing the provider. So you can use something like AWS Bedrock as your Anthropic API or Anthropic itself. And in this case, I just created a proxy container. And my proxy container is actually intercepting the traffic. And by this, I can actually see what the container is doing. The SQLite database is actually on your local machine. And this way, you have your own system where you can track what the container is doing. We have a project workspace. The project workspace is actually the code that is mounted from your machine. And we have another dedicated volume where we are storing the whole configuration files that are required for the agent. So in this case, if you want to log into the system, you will stay logged in if you run another container. What it didn't have in this iteration was also tracking things like calls to search tools or other websites. This was not part of that project. OK, so the idea is we keep everything separated in a container. We route the traffic to the proxy. And this way, at least we can isolate those failures to the machine itself. So if we have something bad, if a bad application is installed, or if the agent decides to remove things, it's going to be at least isolated to that specific container that we are running. I showed this to a few of my colleagues. And then they said, can we have one for Gemini? Can we have one for Codex? Can we have one for OpenCode? And so on. So I decided to dig deeper into each of those agents, and I created those projects. And soon I realized this is not really easy to maintain now because I now have a different command line for every agent. I don't really want this. Also, for each agent, I now have a dedicated database which is running, and it's really hard to onboard other team members into it. So, I decided to build something totally different on a new approach, and I created this project which is called VibePod. In this project, I tried to generalise the pattern. I created it mainly for making a better experience with the CLI. Before everything was created with Bash, I moved it now to Python. The CLI allows you to use it with all the containers, and we can now reuse the patterns for the proxy and also for the database for logging for all agents. There are some tools that I use for this. The most important are the Typer for creating the CLI, also PlatformDeers for making sure that the configurations run on multiple operating systems. This is how the architecture looks right now. So instead of having just a single proxy that is used for routing the traffic of the Anthropic API, we are now having a man-in-the-middle proxy. This is something that I found for other agents, and it works very well, because like this, we can now really monitor the entire traffic, and we can add everything to our database. Really each individual request. At this point also extended it to also track activity that is made by WebSocket connections. Now also the agents, they have also their own repository, so with time those agents can also evolve and also multiple agents can be supported. How does this look? You have now this VP as the CLI tool, and you can just say VP run cloud, and it will start your cloud instance, and like this, you can use it for all the other agents. Then, if you want to see what the agents are doing, you just have to do the list command. You can just see like this what is running on your system. You can stop them individually, or you can stop all of them if you want. Now it's time for the live demo, and let's hope this works so I have here my system currently so I have the VPE CLI installed and I can see here if I run this VP version I'm going to see what is currently on my system I see the version of the CLI tool I see the version of python which is also important also the version of docker because this is the tool that is currently based to run on and if I now want to run Cloud, I can just say VP run Cloud. And first thing, it will ask you if you give permissions to the specific folder to run. I will say yes. And then it will start my container. Also important, if you try to run it on your home directory, which might happen by accident, this thing will prevent it. Because root directories and home directories will actually be on a deny list, so don't do this by accident to share your entire home files. Okay. Then what we can do now is I will take a simple task from my GitHub repository, and I will just tell the agent to do something. And the whole idea now is that agent will start working in the background, will do some API calls and based on the skills that I have configured to ask me questions and so on. So the important part is now how does the logging look? So we have here our HTTP proxy log, so maybe I will just show you how the database looks. So the tool that I'm using for the visualisation is called Dataset, and it's a tool that is very cool if you're having a SQLite database and want to analyze some data. So the first thing is here we have the proxy database and inside the proxy database you can see you have all your HTTP requests and also all your HTTP responses and also here you see the WebSocket messages. So if I want to go over the HTTP requests I see here I can sort descending and I can now see the API calls that we have just done. And what I've also done here, I showed the responses also, also the requests and responses just as binary so you can download them and you can see the entire prompt that Cloud is actually doing in order to operate. Also here you can see the responses individually, but the most important part is you can now use these data also to create dashboards which are a bit more nice to see so currently there is a HTTP dashboard implemented the HTTP dashboard will show you how many requests your agent is actually doing and you can see here each individual request and you can see also the timing of the request so you can also kind of isolate requests that take longer based on the payload and also you can see failing requests, so in case that the anthropic API is currently not available, you can also track it like this and see when a downtime happened. The other thing that we have are dashboards for the token usage that are currently implemented for Cloud and for Codex. And the token usage here happens a bit differently than on other projects that are available on GitHub. Typically, those dashboards are built using the JSONL files that the agent is storing. What I do here is I use the raw HTTP request, so Anthropic gives me with every response also the token consumption, which means I can aggregate that data and also show it here. So this means I can look at each individual request. I can see how many input tokens I had, how many output tokens I had. I can also see the cache tokens for reading and for writing and also the response time. If I have a response like this here, it probably means that I cut up the agent and the response didn't finish. That's why it's not stored. Also, if you're curious, you can look with dataset into each individual query and you will have some messy SQL here which you can play. You can also edit it live if you want and also see and analyze the data. Okay. Then let's just see how our agent performs exactly so now we have this thing every time you run the agent agent We'll ask you questions If you don't want to do this There is a special mode you can use Which is called. I know what I'm doing and If you start it like this, it's going to directly append the dangerously skip permissions for you you. And the idea was, every agent is calling that mode differently. So, you have one syntax for Claude, you have one for Codex, for Gemini, I think on Gemini it's called the YOLO mode. And I wanted to just have one parameter, and I wanted to kind of group them so you don't have to think about what's the parameter actually called. Yes, and like this, I can rerun the the prompt, and it will just go through and will auto-approve. It will only stop if there is really some direct input needed. If the agent considers that there is not sufficient input, it will ask me for text input. OK, then let's just see where we are. So what we did in demo flow quickly, so we looked at an isolated agent run. We just ran a small task, and we looked a bit at the data dashboard. One quick thing that I forgot to mention, the dataset dashboard is actually also running using the VPCLI, so all you have to do is run VPUI and this will start your container and will automatically open in the browser. So if you want to install it yourself, here is a quick run, you can just do it with If you are on a Mac, you can do it with this long command. It's actually a private tab, that's why it's so long. Then you can go into your project, run VP list just to see what's available, and with the VP run and the specific agent, you can see the agent. You can either use the shorthand, the VPUI, or you can just say the long command VP logs start and this will start your logging service so what you can also do I forgot to show you the VP command itself if you run just this you can get the help of the command you can also install all the completions if you want and you also have the VP list command which is very important because this one we show you what agents are available on your system and it will also show you on which directories you're currently having an agent running, and it will also show you the context that is actually bound to your agent. And this data by itself, it's also available in Dataset. If you go into the logs, you can also see here you have the sessions, and here on the sessions you can see which agent you used and to which directory you bound it, and also with which version of the CLI you were working. And also another thing, it's also collecting the messages that you type into the agent. But this doesn't work perfectly now because the agent also has some sort of artefacts in the CLI. So this works kind of okay, but we use it with caution. So just continue. There are some limits, however, which you should consider. The container will reduce the attack vectors, but it will not completely protect you. So you're really not protected to prompt injection attacks. So the things that I've shown in the scenarios, if you, by accident, get some sort of MPM package installed, this will still happen, but it's isolated to your container. If you are misconfiguring the mounts and you have some critical files in your context, you will also expose those information, so be careful. And the logging will actually help you find how the agent works, but this is not a prevention. It's just for history purposes. So what's important is you should still do human review and should make sure that you have right policies in place. So main takeaways, try to use those agents, isolate them as good as possible, give them only the context that they need to work by applying a least privilege to files, networks, credentials, and also from time to time, make sure that you look over what they are doing and try to measure on your observability part what actually matters. I have some resources here to the different projects that are available, and from here I'm ready for your questions.

Speaker 1 [22:06]

We have a few questions in the chat, and maybe we can take some from the audience. How do you handle credentials and login for hyperscalers?

Speaker 2 [22:17]

I

Speaker 1 [22:18]

i.e. one agent credential per container.

Speaker 2 [22:21]

Okay, so what you can do is, since you're running them individually, I typically work with Docker containers also for my application. So why I end up is I have an environment that I set up for my application containers, and then I have the container with the agent itself, so they are separated. And when I run my environment for the applications, it's loaded just with the credentials that are needed for the application to run. and my agent doesn't see those environment fires.

Speaker 1 [22:55]

Is there a minimal VP vision without all the monitoring, termitory overhead if all I care about is the isolation?

Speaker 2 [23:07]

If you only care about the container, you can go into the VP agents repository, and there are all the images also. So, in fact, let me show you this. Down here in the documentation, you can see the pre-built images. So, if you just want to run the container by itself, you can do it like this directly. The VPCLI is actually taking care of creating the Docker network, connecting your container with the agent, with the proxy container, and also making sure that all configurations are in place for the tracking.

Speaker 1 [23:42]

Okay, how does VP compare with the under-sandboxing projects like Anthropic Experimental or Sandbox Runtime or Docker Sandboxes? Yep.

Speaker 2 [23:52]

Yeah, so it is in fact the Docker sandbox and the thing is you can manage it yourself. What you can also do here is since you are able to overwrite the images, you can just start from that specific pre-built image or you just create your own image as you want it. You can also install all the tools that are required and you can pretty much reuse it over your projects.

Speaker 1 [24:21]

Okay, can you run remote containers into Kubernetes with Phoebe?

Speaker 2 [24:26]

haven't thought about that maybe I don't know

Speaker 1 [24:33]

Are all sub-agents also isolated and monitored?

Speaker 2 [24:42]

Can you repeat that question, please?

Speaker 1 [24:44]

Are all sub-agents also isolated and monitored?

Speaker 2 [24:48]

Sure. If you want to run sub-agents, you're basically running those agents inside the container.

Speaker 1 [24:48]

Sure.

Speaker 2 [24:56]

So if you are starting Cloud Code, Cloud Code will be in a container. If you have the Teams function enabled or simply using the sub-agents, this will be by default enabled, so Cloud will start those sub-agents or sub-processes inside the container.

Speaker 1 [25:13]

Okay, Kano, should you also use dedicated Git repo for the agent so you can keep it separate from your own work in progress and maybe then it doesn't have the remote configured?

Speaker 2 [25:30]

I'm not sure if I understand the question correctly. Yeah, please. Can I get a part from your repository just for the agent? Sure. So as I said, all those projects here are individually available. So this means there is also the source here for how you can build the agents itself. So you can look over the source here and you can find the Docker files for each individual agent. and you can start from here if you want.

Speaker 1 [26:02]

okay does your MITM proxy support or is it post is it possible to support approving or denying network request would be very useful to control exactly what request it is are out to make yeah

Speaker 2 [26:26]

Yeah, I was thought about this. It's a bit of a trade-off now between easy handling and kind of having now, I don't know, the whole network operations tech on your local machine where you are handling whitelisting and blacklisting. If you have any ideas how this would be easy to manage, I'm happy to discuss because I want to introduce something like this, but I don't want to have that much overhead.

Speaker 1 [26:53]

Okay, can you still use plans like cloud code with your MIT proxy, or do you need to use paper use API?

Speaker 2 [27:02]

Oh, no, you can you can use the the regular plans. So so in this case here, in fact, I'm using a pro plan so You see this when you start the agent because the tool itself. It's not changing the agent you have the the agent as provided by the vendor installed in your container and you're Just having the tool to provision the container for you and forward the the request So that's all it's doing at the end. It was just for convenience. So we have it for for all the agents in the same way but as you see here, I have the cloud pro subscription and there is absolutely no problem with it because There they don't change the requests themselves that are sent by the agent

Speaker 1 [27:48]

Maybe we can take two questions from the audience, if there is anyone who would like to ask.

Speaker 2 [28:04]

Is there an easy way to integrate it with pre-existing dev containers, for example the monitoring spec? So, you have the containers already in place, you should try it, honestly. I'm not using myself that much on dev containers, I'm more on that part where I build my compose configurations and I'm happy with that but you can try it yeah on Mac says your new application container okay would it be possible to replace docker vices is it easily or is it it has some yeah the the whole thing is I also try also to It's a Mac architecture. I see. The important part is, does it have a Python API exposed so we can communicate with it? If it has that in place, we might have a starting point. The other part is maybe the permission handling. I'm also trying for some time to make this work with Podman, and Podman is a bit more strict on permissions on your files, so I kind of want to also finish and integrate this to make it more agnostic. So you can also run it with different container runtimes. But sure, let's just talk after the session. Maybe a quick question. So how much of the WipePod is actually WipeCoded? A lot. A lot. So I have to say the first project, the Cloud Container, I built it from scratch and I built it manually. And afterwards, I actually decided I want to have this thing in place with the CLI since I had all the learnings and the patterns from the previous projects I actually reused them and I actually also used all the projects as reference.