The Cost of an Idle Agent

7 min read

B25
Operation · Economics & ROI

The cost of an idle agent.

The moment a run outlives a single request, your bill stops measuring work done and starts measuring resources held — and the longest hold in most agent systems is a human approval that nobody costed. A 2 vCPU / 4 GiB sandbox left alive through a four-hour review wait bills about 66 cents of pure nothing per run; at 200 runs a day that is roughly $4,000 a month for seconds in which no token was generated. The lever is not the prompt. It is whether anything is held open while nothing is happening.

Step 1Long-horizon runs change the unit of the bill.

Token pricing is a cost per unit of work: you pay for what the model read and wrote, and an agent that does nothing costs nothing. Every instinct in agent cost control — fewer tokens, cheaper tokens, caching, routing — is tuned to that meter, and all of it keeps working.

It stops being the dominant meter the moment a run spans more than one request. A long-horizon agent holds things between steps: a sandbox with a checked-out repository and a warm virtualenv, an open audio stream, a reserved slice of provisioned throughput, a browser session with a logged-in profile. Those are priced per unit of time, and time passes whether or not the agent is thinking.

The result is a cost structure most teams never model. Agent unit economics gives you cost per task; what follows is the term in that number which does not scale with tokens, does not respond to a better prompt, and grows linearly with how long your agent waits.

# two meters, two units
tokens      $ / 1M tokens        → charged for work performed
sandbox     $ / vCPU-second      → charged for time held
audio       $ / second of audio  → charged for wall-clock
reserved    $ / hour committed   → charged whether used or not

# an agent that is waiting has a token bill of zero
# and a wall-clock bill identical to one working flat out.

Step 2"Alive" is the pricing term, and vendors define it differently.

Agent sandbox providers look close to identical on a rate card and differ by an order of magnitude on an idle-heavy workload, because the rate card prices a vCPU-hour and the bill prices a lifetime.

  • Billed while alive. E2B and Daytona meter CPU and RAM per second for the whole time a sandbox exists — not only while code is executing. Third-party comparisons published through 2026 put both near $0.0504 per vCPU-hour and $0.0162 per GiB-hour, with Daytona adding a small storage charge around $0.000108 per GiB-hour.
  • Billed while running. Modal meters max(requested, actual) per second while the sandbox runs and charges zero compute while it sits idle. The same comparisons list roughly $0.00003942 per physical-core-second plus about $0.0000067 per GiB-second, and because one billed core is two vCPUs that is a higher per-vCPU rate than E2B or Daytona while actually executing.

So the cheaper per-hour vendor is the more expensive vendor whenever your sandboxes spend most of their lives waiting, and the direction of that inversion is set by your workload, not by the price list. One published model of an idle-heavy profile puts a 2 vCPU / 4 GiB sandbox near $36 a month on the pause-while-idle meter against roughly $414 on the billed-while-alive ones.

Those rates are third-party figures from comparison write-ups, not quotes we could trace to a vendor pricing page, and they move. Use them as the shape of the bill — one meter stops when nothing runs, the other does not — and re-derive the arithmetic from the vendor's own page before you commit to a provider. The shape is the durable fact; the decimals are not.

Step 3The approval wait is the largest idle block you own.

Idle time in an agent system is not mostly jitter and network. It is mostly a person. A run that pauses for sign-off before a write, a payment or a send — the whole design in human-in-the-loop and least privilege — waits on a review queue whose latency is measured in hours, not seconds, and gets worse out of hours.

Put numbers on it with the billed-while-alive meter above. A 2 vCPU / 4 GiB sandbox costs 2 × $0.0504 + 4 × $0.0162 = $0.1656 per hour:

per-run hold         4 h × $0.1656/h            = $0.66
200 runs/day                                     = $132 / day
30 days                                          ≈ $3,975 / month

for comparison, the model spend on those runs:
  ~40k tokens in / 6k out at mid-tier rates      ≈ $0.05 / run
  200 runs/day, 30 days                          ≈ $300 / month

The waiting costs thirteen times the thinking. Nothing in that larger number bought a single token, and no prompt change touches it. Note also what it is not: the reviewer's own time is a separate and usually larger line, priced in the cost of human review. This is the bill for the machine that sat still while they got to it.

Two secondary meters tick during the same wait. A prompt cache has a TTL, so a pause longer than the window converts a resume into a cache write at its premium rather than a hit — the arithmetic is in context-caching economics. And a reserved-capacity commitment bills through the wait by construction, which is what you bought; provisioned throughput and commitments covers sizing it against a duty cycle that includes the gaps.

Step 4Voice is the case where nothing can be parked.

A realtime voice agent bills per second of wall-clock, so dead air, hold music and a caller thinking cost exactly what speech costs — the four-meter breakdown is in the cost of a voice minute. What matters here is that the usual fix is unavailable: you cannot suspend a live call.

The providers also cap the session, which turns "park it and come back" into an architectural impossibility rather than an expensive choice. OpenAI's Realtime documentation puts the maximum duration of a session at 60 minutes, raised from an earlier 30. Google's Live API caps audio-only sessions at about 15 minutes and audio-plus-video at about 2 without context-window compression, and individual WebSocket connections time out around 10 minutes, with session resumption disabled unless you ask for it in the setup message.

So on a voice run, the only idle lever is to shorten the idle: demote the model while waiting, cap the wait with a defined action on expiry, and hand off rather than hold. Realtime agent architecture has the structural version of the same constraint.

Step 5Releasing beats holding by two orders of magnitude, and money is not why teams hold.

The break-even looks like it should be a judgement call and is not. At $0.1656 per hour, one second of hold costs about $0.000046. A three-second cold start consumes the same metered resources for three seconds and therefore costs about $0.00014. Holding is cheaper than restarting only for waits shorter than the cold start itself — a few seconds. Every wait measured in minutes should release.

Teams hold anyway, and for two reasons that are both real and neither of which is cost:

  • State. The sandbox is the run's memory — the working tree, the installed dependencies, the half-finished edit. Releasing it means you need the run to be reconstructible, which is the whole subject of durable state and resumability. A team without that cannot release, so it pays rent on a VM to avoid writing a checkpoint.
  • Latency. Resume has to feel instant to whoever approved. That is a pool-sizing and snapshot problem, priced in sandbox pools and cold starts — and a pool sized for peak is itself an idle bill, so the two fixes trade against each other rather than compose.

Which makes the design rule concrete: make waiting a state of the run rather than a state of a machine. Checkpoint the run, release the sandbox, let the approval land asynchronously, and rehydrate on whatever host is free. The UX half of that — telling a user their job is paused without a spinner holding a connection — is async agent UX. The deploy half, where a long-lived session outlives the version of the code that created it, is long-lived sessions and deploys.

Step 6Meter held seconds separately, or you will never see this line.

The reason idle cost survives in production budgets is that no standard dashboard distinguishes it. Spend per run, spend per customer and spend per model all aggregate held time into the same bar as working time, so the number looks like usage growth. Three changes make it visible:

  • Split billed seconds into executing and held, per run, and emit both as span attributes so they land in the same trace as the tokens. Cost attribution and budgets is where they then roll up.
  • Track held share, not held dollars. Held seconds divided by total billed seconds is a ratio you can set an objective against; it rises when review queues back up and when a pool is oversized, and it does not move when traffic grows. Dollars confound the two.
  • Alert on the longest hold, not the average. Idle cost is a tail statistic: a handful of runs abandoned with a live sandbox will outweigh thousands of clean ones. A maximum hold duration per run, enforced with a reaper, is the cheapest control in this essay.

If the ratio is above roughly a third, the agent's dominant cost is not inference and no model-level optimisation will show up in the invoice. The levers are the queue, the checkpoint and the reaper. Fixed costs and the pilot tax covers the related trap of a reserved footprint sized for a pilot that never arrived.

Start by measuring one number: billed seconds held divided by billed seconds executing, over a week of real runs. If it is under about 0.2, stop here and go optimise tokens. If it is over 1 — more waiting than working, which is the normal result for any agent with a human gate — do three things in this order. Put a maximum hold duration on every sandbox with a reaper that enforces it, which caps the tail tonight. Then move the approval wait outside the hold: checkpoint the run, release the sandbox, resume on a fresh one. Only then compare providers, because the idle policy, not the per-vCPU rate, is what you are actually buying.