Your Agent Needs Your API Keys but You Can't Trust It with Them

2026-03-03

by Uri Walevski

Give an agent an API key and it will leak it. Not maliciously, not on purpose. It'll echo it in a debug log. It'll paste it into a tool call that sends it to the wrong service. A user will social-engineer it out with a well-crafted prompt. The key will sit in plaintext in the agent's memory, waiting for something to go wrong.

This is the fundamental tension with useful agents. To do real work, your bot needs credentials. To push to GitHub it needs a token. To query your database it needs a connection string. To deploy your app it needs cloud provider keys. But the moment you hand those over, you've created a security surface that scales with every conversation, every prompt injection attempt, every bug in your tool chain.

Most platforms deal with this by not dealing with it. They either don't give agents real credentials, which limits what agents can do to toy demos, or they inject secrets as environment variables and hope for the best.

We took a different approach with prompt2bot.

The Agent Never Sees the Secret

When you add a secret to your bot, you give it a name, a value, and a list of hostnames it's allowed to be used with. The value is encrypted immediately. It's never shown again, not even to you.

The bot knows the secret exists. It knows the name and which hosts it's for. But it never has access to the actual value. Not in its context window, not in its environment, not anywhere in the VM it runs on.

When the bot makes an outbound request to an approved host, the real credential gets injected at the network level. The bot's code runs normally, it just never touches the real key. If someone prompt-injects the bot into dumping its environment, there's nothing useful to dump.

I'm not going to detail exactly how this works. But the result is: the secret value exists in exactly one place (encrypted in our database), and only appears in cleartext for a fraction of a second, at the moment it leaves our infrastructure toward the target API. The VM, the container, the agent, the LLM, none of them ever see it.

Why This Matters More than You Think

Without this, you're basically choosing between two bad options. Either your agent is a toy that can't do anything real, or it's a liability holding your production credentials in a context window that any creative user can extract.

There's a third problem nobody talks about. Agents make mistakes. A bot might call the wrong tool with the wrong arguments. If your GitHub token is a string the agent is passing around, it might accidentally send it as a parameter to your Slack integration. With our approach, even the agent's own mistakes can't leak the secret, because it literally doesn't have it.

Secrets in Conversation

Users paste API keys into chat all the time. "Here's my GitHub token, set it up." If you store that message in conversation history, the key is now in every future LLM call for that conversation.

We detect secrets in incoming messages automatically. GitHub PATs, OpenAI keys, AWS credentials, JWTs, long hex strings. They get replaced with a placeholder before the message ever reaches the model. The bot sees <<SECRET_xxxxxxxxxxxx>> and knows to store it, but the actual value is already encrypted and gone from the conversation.

The VM

The bot gets a real virtual machine. Not a stateless Lambda, not a sandbox that resets every call. A persistent server with Docker, a filesystem, installed dependencies. Your repo stays cloned between conversations. Your tools stay configured.

The VM is where the bot does its actual work: cloning repos, running tests, reading logs, opening PRs. It's also where a coding agent runs on behalf of the bot, handling complex multi-step programming tasks.

Each bot gets its own VMs, its own secrets, its own isolation boundary. Bot A can't see Bot B's machines or credentials, even if they run on the same infrastructure.

What This Enables

A bot that can manage your GitHub repos without you worrying about your token ending up in a prompt injection blog post. A production engineer bot that checks your logs at 3am using real cloud credentials that it can't leak. A deployment bot that pushes to your staging environment without the deployment key ever existing in memory on any machine the bot controls.

The security model isn't "we trust the LLM to be careful." It's "the LLM is structurally unable to access the credential." That's a different category of guarantee.

← All posts