When the Credential Broker Is Compromised

Step 1Concentration is the price of the pattern, not a flaw in your implementation.

Work through why the broker exists and the concentration is unavoidable. An agent reads attacker-influenced text on every step, so any secret reachable from inside the loop is one crafted instruction away from a tool argument or a URL — the argument secrets management for agents makes in full. Moving the secret out means something else must hold it, and that something must sit in the path of every call, because a credential that is not attached at egress is not attached at all.

So you have traded a diffuse risk for a concentrated one, and the trade is still worth making: a leaked trace, a crash dump and a chatty error can no longer contain a key, and those are the common failures. What changes is the shape of the worst case. Before, a prompt injection reached one credential the loop could see. After, a compromise of one host reaches every credential the broker was configured with — and nothing about the second is mitigated by the control that fixed the first.

The LiteLLM compromise is the concrete version. The malicious 1.107.0 release dropped a stealer that ran at every Python interpreter start, and Datadog Security Labs' analysis lists, among its targets, the upstream provider credentials stored in the proxy's database where one was configured. Not the model's context. Not a leaked log. The broker's own store, read by code running beside it. We covered the supply-chain mechanics separately in the pipeline was real, the fork was not; what matters operationally is that the component holding your keys is a component with a release channel.

Step 2Count the union, then say it in one sentence.

Nobody funds work on this until the number is written down, and the number is usually larger than the team's estimate because credentials accrete. The measurement takes an afternoon: enumerate what your broker can present, not what your agents currently call.

• Distinct credentials held. Every provider key, every internal service token, every database credential the broker itself uses. Include the ones added for an experiment that ended.
• The widest single grant. Which held credential authorises the most, and at whose provider. One organisation-wide key with billing authority dominates the risk of twenty narrow ones.
• Tenants reachable. If the broker serves more than one customer or team, whether a compromise crosses that line. This is where multi-tenancy for agents keeps finding the isolation was drawn around the wrong object.
• Secondary stores. The broker's database, its config file, its environment, its backups and its logs. A credential that appears in four places has four blast radiuses, and backups are the one people forget.

Then write the sentence, in the same shape that works for every other authority question: a compromise of this process yields N credentials across M providers and T tenants, the widest of which can do X. That sentence is what gets the work scheduled, and the blast radius framing is the only one most executives will act on.

Step 3Hold the narrowest thing each provider will issue you.

The single highest-value change is upstream of your own code: stop holding long-lived, organisation-wide provider keys and hold whatever narrower, shorter-lived credential each provider actually offers. The options differ by vendor and they move, so the durable instruction is a question to ask rather than a feature list — for each provider in your broker, what is the narrowest credential available, scoped to which project or workspace, with what expiry and what spend ceiling?

What you are buying is a bound on the worst case that does not depend on your detection. A key scoped to one project with a spend cap limits a compromise to that project and that amount, whether or not you noticed. A root key with billing authority limits it to how fast you can act, which in a real incident is hours. This is scoped credentials for agents applied to the broker itself, and the inversion from that page holds here too: scope first, lifetime second.

Lifetime still matters, but for a narrower reason than usual. A short TTL is weak against an agent that is already running next to the credential — the argument credential lifetime makes. Against a broker compromise it is stronger, because the attacker's usual move is to exfiltrate the store and use it elsewhere, later. That is exactly the offline adversary short lifetimes were designed for. So expiry does real work here that it does not do one layer in, which is worth knowing because the two layers get discussed as though the same reasoning applied.

Three further shapes, in descending order of how often they are skipped:

• Per-tenant credentials where the provider allows them. The difference between a shared key and a per-tenant key is the difference between one incident and one customer's incident.
• No credential in the broker's environment or config file. Reference them from a secret manager the broker reads at runtime, so the store the stealer finds is a set of names.
• A spend ceiling on every key, set low enough to notice. Providers meter; use it. A cap is the one control that works while everyone is asleep, and denial-of-wallet is a real outcome of a stolen model credential rather than a theoretical one.

Step 4The broker's release channel is now part of your security boundary.

A component that holds every provider credential you own should not be upgraded the way a logging library is upgraded, and in most teams it is. Three changes, all cheap, all specific to the fact that this process is a credential store:

• Pin by digest, not by tag. Immutable image digests and hashes in the lockfile. The October incident's exposure window was 101 minutes on PyPI and 134 for the container images; a floating latest or main-stable is what converts a two-hour window into your incident. Pinning and verification is the general case, and note the specific trap: a provenance attestation verifies the pipeline, not the source, so it would have passed that release.
• Stage the broker's upgrades behind everything else. Not because it is fragile, but because the cost of being early on this component is the union from STEP 2. A deliberate lag of a few days on a credential-holding component is a security control with a real, small cost, and it is the cheapest one in this list.
• Treat anything that can write to the broker host as release authority. Developer CLI tokens were the initial access in October. The accounts and tokens that can deploy or exec into this host are, in effect, holders of every credential it has.

And keep the gateway's own role in view: if you are self-hosting, you own this upgrade path, which is both the burden and the point. A hosted gateway moves the same concentration to a vendor whose release pipeline you cannot pin at all — a real trade, not an obvious win either way, and AI gateways sets out what the component does for you in exchange.

Step 5Rotation is a capability you either have in minutes or do not have.

Every team believes it can rotate. Almost none has measured it, and the measurement is the deliverable here: pick a non-production broker, declare every credential it holds compromised, and rotate all of them while someone holds a stopwatch. The number you get is your actual exposure after detection, and it is usually dominated by two things nobody designed.

• Finding the credentials. If rotation begins with a search — which provider consoles, which projects, who created this key three quarters ago — you do not have a rotation capability, you have an investigation. The fix is the inventory from STEP 2, kept current, with an owner per credential. Access reviews for agent credentials is the governance habit that keeps it from rotting.
• Reloading without a redeploy. If new credentials require a rebuild and a deploy, rotation inherits your release process on the worst day to inherit it. The broker should re-read its secrets on signal or on a short interval, and that should be tested, because the failure mode is a half-rotated fleet serving errors.

Two rules that only matter under pressure, and are both routinely skipped:

• Rotate from a machine that was never exposed. Rotating from the compromised host hands the replacements to whatever is reading that host. Obvious written down; genuinely forgotten at 02:00.
• Revoke, do not just replace. Issuing a new key does nothing to the old one. The old one is the one the attacker has. Replacement and revocation are separate actions at most providers, and only the second ends the incident.

Then audit use during and after the window, because rotation closes the future and tells you nothing about what already happened. Provider-side usage logs are the record that matters, and they are a different system from your own traces — which is the practical reason incident response for agents wants the access path written down before you need it.

Step 6The broker is also the best detector you will ever have. Use it.

There is a compensation for the concentration, and most teams never collect it. The broker is the one place in the stack where every model call is visible with its destination, its tenant and its cost, before it leaves your network. That makes it the highest-signal, lowest-volume detection surface you have, and the detections are the rare kind that are nearly free of false positives.

• An unexpected destination from the broker host. This process should talk to a short, enumerable list of provider endpoints and nothing else. Anything outbound to a host not on that list is a finding, not an anomaly score. In October the exfiltration went to a lookalike domain on a path named /healthcheck — chosen to read as noise, and trivially caught by an allowlist that names destinations instead of blocking known-bad ones. Egress control for agents is the implementation.
• A credential used by a caller that has never used it. The broker knows which agent, tenant and tool legitimately present each credential, which makes first use by anything else a high-signal event. Feed it into detecting agent compromise rather than a dashboard nobody reads.
• Spend velocity per credential. A stolen model key monetises as volume, so a rate that is unremarkable monthly and absurd per minute is the clearest early signal of theft you will get.

One decision to make deliberately rather than discover during an outage: a broker in the critical path has to fail one way or the other, and for a credential broker the answer is almost always closed. If it fails open, either your agents proceed without credentials — which simply fails the calls — or something in the path falls back to a credential held elsewhere, which quietly undoes the whole pattern. Fail-closed and fail-open makes the general argument; the broker-specific version is that a fallback path with its own key is a second broker you forgot you were running.