The Per-Step Cost of Looking

Step 1The bill is denominated in looks, not in prose.

A text agent's input is mostly tokens you can read. A GUI agent's input is mostly an image, and the image has a published price in tokens. Anthropic's computer-use tool documents the conversion exactly: a screenshot costs ⌈width / 28⌉ × ⌈height / 28⌉ visual tokens, with the current toolset capping a single image at 4,784 of them and 2,576 pixels on the long edge. That formula is the whole cost model, and it is worth evaluating once on your own resolutions before anybody argues about model pricing.

# visual tokens = ceil(w / 28) * ceil(h / 28)

1024 x  768   ->  37 x 28  =  1,036    # docs recommend this for desktop
1280 x  800   ->  46 x 29  =  1,334    # docs recommend this for web
1366 x  768   ->  49 x 28  =  1,372
1920 x 1080   ->  69 x 39  =  2,691    # docs warn against going above
2576 x 1448   ->  92 x 52  =  4,784    # the documented per-image ceiling

Two things fall out immediately. The documented recommendations sit at about a fifth of the ceiling, so the ceiling is headroom rather than a default — a point the tool-version split makes from the other direction, since the earlier tool version caps at roughly 1.15 megapixels and cannot reach these numbers at all. And the full-frame ceiling costs 4.6× what the recommended desktop resolution does, on every single step, which is a larger multiple than any model price cut in the last two years.

Step 2The naive loop is quadratic, and the exponent is the problem.

Pricing one screenshot is the easy half. The expensive half is that an agent loop re-sends its transcript on every step, so a screenshot taken at step 3 is an input token at step 3 and again at steps 4 through N. Count the image reads rather than the images.

# N = 40 steps, one screenshot per step, nothing dropped
# step i sends images 1..i, so total image reads = N(N+1)/2

reads         = 40 * 41 / 2            =    820 image reads
tokens        = 820 * 1,036            = 849,520 visual tokens
# 40 screenshots were taken. 820 were paid for.

That is the arithmetic behind the sense that GUI agents are disproportionately expensive, and it is the same quadratic that governs text loops — see agent cost control — except that the per-item constant is 1,036 tokens instead of a 40-token tool result. The images are not a line item alongside the transcript. On a long task they are the transcript.

Step 3Prompt caching turns the quadratic back into a line.

The 820 figure assumes every re-read is billed at the full input rate. With prompt caching and an append-only transcript it is not: the prefix is a cache read, and only the newest image is written. Using the standard multipliers — 1.25× to write a cache entry, 0.1× to read one — the same task re-prices by a factor of six.

# Same 40 steps, caching on, history append-only.
# Step i: read images 1..i-1 from cache (0.1x), write image i (1.25x).

cache reads   = 0.1 * (0 + 1 + ... + 39) = 0.1 * 780 =  78 units
cache writes  = 1.25 * 40                            =  50 units
total                                                = 128 units

# versus 820 units uncached -> 6.4x cheaper, same screenshots,
# same resolution, same model.

This is the first thing to fix and it is usually free, because a computer-use loop is already append-only in shape: perceive, act, perceive again. The reason it so often is not cached in practice is that the harness mutates history for unrelated reasons — injecting a reminder, re-ordering blocks, rebuilding the message list from a database each turn. Each of those invalidates the prefix and silently restores the quadratic. The mechanics of which edits cost what are in context-caching economics and KV-cache budgeting for agent loops.

Step 4Trimming old screenshots is the wrong lever, and it is worse than nothing.

The intuitive saving is to keep only the last few frames. Run the numbers on it and the intuition collapses, because dropping the oldest image changes the prefix on every step, which is exactly the edit that costs you the cache.

# Keep only the last 3 screenshots, N = 40.
reads  = 1 + 2 + 3 * 38          = 117 image reads

# (a) you had no cache:   117 units  vs 820  -> 7.0x better
# (b) you had a cache:    117 units  vs 128  -> 1.09x better
# (c) you trim AND pay cache writes on a prefix that
#     changes every step: 117 * 1.25 = 146 units -> WORSE than 128

So the ranking is not what the planning meeting assumes. Against an uncached loop, trimming looks like a 7× win and caching like a 6.4× win, and they appear interchangeable. They are not, because they do not compose: a trimmed history cannot be cached, so you get one of them, and the one that also preserves the agent's ability to compare the current screen against three steps ago is the cache. Case (c) is the one actually shipping in production somewhere right now — a team that enabled caching for the text prefix and then added image trimming as a second optimisation, paying the write premium on a cache that never hits.

Step 5Buy detail with a zoom, not with a bigger frame.

Resolution is not purely a cost question, which is what makes it the hard one. Spatial grounding — producing the coordinate rather than naming the target — is the dominant error class in GUI agents and it degrades as you downscale, so cutting the frame to save tokens buys a cheaper step and a less accurate one. The current toolset resolves that trade rather than splitting it: capture the frame at the recommended size and re-capture only the region you are about to act on.

# Full frame at the ceiling, every step:
2576 x 1448                            = 4,784 tokens / step

# Recommended frame, plus a zoom on a 400x300 region:
1024 x  768   ->  37 x 28  =  1,036
 400 x  300   ->  15 x 11  =    165
                             -------
                               1,201 tokens / step

# 4.0x cheaper, and full resolution exactly where the click lands.

Two cliffs sit next to this and both are easy to walk off. The first is the documented image-count threshold: once a request carries more than 20 images, the per-side limit tightens to 2,000 pixels — so a full-retention loop changes its own observation contract at around step 21, which will read as an unexplained accuracy drop halfway through long tasks. The second is coordinate scaling. Coordinates come back in screenshot pixels, so if you downscale before sending you must scale the model's answer back up, and a rounding error there is a click that lands near the button rather than on it. That turns a cost decision into a failure mode, which is why the resolution lever deserves less enthusiasm than its multiple suggests.

Step 6Measure it per step, not per task.

The unit that makes this manageable is visual tokens per step and steps per completed task, held separately. They move for different reasons and they respond to different fixes: tokens per step is a harness setting you control in one place, while steps per task is a competence number that changes when the model or the target application changes. Reporting only the blended cost per task hides which one regressed, and on this surface they regress in opposite directions — a sharper frame raises tokens per step and lowers steps per task, and the sign of the net effect is an empirical question about your targets.

Three numbers are worth putting on a dashboard and keeping there. Visual tokens per step, computed from the formula rather than estimated, so a resolution change shows up the day it ships. Cache hit rate on the image blocks specifically, because a global hit rate stays healthy on the text prefix while the expensive half misses. And image reads per completed task, which is the quantity the quadratic acts on and the one that catches a retry storm turning a 40-step task into a 90-step one. Record all three against the model and tool version, as cost attribution requires, and reconcile against the provider's returned usage object rather than your own tokenizer, because visual tokens are not something you can recount locally.