Files as the Context Window
Writing the observation to a file does not take it out of the loop. It only makes the re-read optional — and every token you save is saved in the slice, not in the file.
The cheapest way to make an agent cost more is to offload a large tool result to disk and then let the agent read the whole file back on every turn: you now pay the same tokens you were paying inline, plus a round trip per turn, plus a worse cache hit rate. Offloading is not compression. It converts a mandatory carry into an optional read, and the entire saving lives in two numbers nobody instruments — how often the file is re-read, and how much of it is read each time. Get those right and a 48,000-token observation costs 12,000 tokens over a 20-step run instead of 816,000. Get them wrong and you have rebuilt the transcript with extra latency.
Step 1What a file actually buys you, stated precisely.
The pattern is now standard practice. Manus's engineering write-up describes treating the file system as the agent's primary long-term context, on the grounds that files are effectively unbounded, persist across steps, and can be operated on directly; large observations go to disk and the context keeps the path. Claude Code reads project files on demand rather than pre-loading a repository. Agent Skills ship as directories the model opens when it needs them. The mechanism is the same in all three cases, and it buys exactly three things.
- Reversibility. Truncating a 48,000-token tool result to its first 2,000 characters destroys information. Writing it to
./work/api-dump.jsonand keeping the path does not — the agent can go back. This is the property that distinguishes offloading from the lossy techniques in context compaction, and it is the one that actually matters, because the thing you drop is reliably the thing you needed at step 14. - Selectivity. Inline content is paid for on every subsequent turn whether or not it is used. A file is paid for only when read, and only for the bytes read. That is the whole economic argument, and it is conditional on the agent reading selectively — which is not automatic.
- Durability past the session. The transcript dies when the run ends; the directory does not. A file is the only one of the two that survives a compaction, a crash, a handoff to a sub-agent, or tomorrow.
What it does not buy you is a smaller problem. The bytes still exist, the model still has to read them to use them, and reading them still fills the same context window described in context budgeting. Nothing about writing data to disk makes a model better at attending to it — see effective versus advertised context for what happens when you put it all back.
Step 2The arithmetic, because the size of the effect is the argument.
Take one concrete run. A tool returns a 48,000-token JSON payload at step 3 of a 20-step task. Models are stateless, so anything left in the transcript is re-sent on every following step — the quadratic effect described in agent cost control. Four strategies, same payload, same task:
# 48,000-token observation at step 3 of a 20-step run. # "paid" = times those tokens enter a request's input. # assumes a read is evicted from the transcript after its turn. # strategy reads paid input tokens inline, left in the transcript 0 17x 816,000 offloaded, full re-read every turn 17 17x 816,510 # + 17 round trips offloaded, full re-read 4 times 4 4x 192,510 offloaded, grep then 2k slice, 6x 6 0.25x 12,510 # pointer overhead: ~30 tokens x 17 turns = 510 tokens, all four rows. # the 98.5% saving is not from the file. it is from reading 2,000 # tokens instead of 48,000 — the file only made that possible.
Read the second row carefully, because it is the one teams ship by accident. An agent instructed to "check your notes before each step" and handed a read_file tool with no range argument will do exactly that: pay the full inline cost, add a tool call and its latency to every step, and — because the file now arrives fresh in a different position each turn — degrade cache behaviour as well. The offload was strictly negative. The only row that wins big is the fourth, and it wins because the agent had a way to read 4% of the file.
So the design question is not "should large results go to disk". It is "can the agent locate the part it needs without reading the whole thing". If the answer is no — an opaque blob, a minified payload, a 600-page PDF with no structure — offloading buys you reversibility and nothing else, and you should budget for it as if the content were still inline.
Step 3A path is not a pointer. A pointer is what lets the model decide without reading.
If the transcript says wrote output to ./work/tmp7.json, the model has no basis for deciding whether to read it, so it will either read it defensively every turn or forget it exists. Both are failures, and they are failures of the pointer, not of the agent. A usable pointer carries four things in under about fifty tokens: what the thing is, how big it is, what shape it has, and how to address a part of it.
# useless wrote output to ./work/tmp7.json # usable: name, size, shape, and an addressing hint ./work/invoices-2026-q3.json — 48k tokens, 1,840 records, keys: id, vendor, amount_cents, status, paid_at addressable: jq '.records[] | select(.status=="open")'
This is the design that Agent Skills formalise, and the published token budget is the useful part of the specification: Anthropic's own account puts a skill's frontmatter metadata at roughly 50 tokens, the body of its SKILL.md at roughly 500, and bundled reference files at 2,000 or more — with only the first tier present at all times and the deeper tiers loaded when the task actually matches. Third-party sources quote different per-skill metadata figures, so treat the exact number as approximate and the ratio as the point: roughly a hundredfold gap between knowing that something exists and reading it. That ratio is what you are engineering when you write the pointer. The same discipline applied to tool outputs rather than instructions is shaping tool results, and the question of which mechanism should carry which knowledge is skills vs subagents vs MCP vs hooks.
The corollary is that the tool which writes the file should also provide the tool which slices it. Handing an agent read_file(path) and hoping it writes the right grep is how you get row two of the table above. Give it read_file(path, offset, limit), a search primitive, and — best of all — the ability to run a query against the file in code rather than in tokens, which is the argument of code as action: a jq filter that returns 40 records never puts the other 1,800 in the context window at all.
Step 4Where the file content lands in the prompt decides whether you saved anything.
Offloading interacts with prompt caching, and the interaction is sharp enough to reverse the sign of the whole exercise. A cache hit requires an exact prefix match from the first token, and cached input is billed at a small fraction of base rate. So the position of your file content is a pricing decision:
- Appended at the end of the message sequence — as a tool result, where tool results go. The system prompt, tool definitions and earlier turns are untouched, so the cached prefix still matches and you pay base rate only for the new bytes. This is the only shape that works.
- Re-injected near the top — the pattern where a
NOTES.mdorstate.jsonis spliced into the system prompt and rewritten each turn. Every rewrite changes a token early in the prefix, which invalidates the cache from that point forward, which means you pay full rate for the entire stable preamble on every turn. On a 12,000-token preamble over 20 turns, that is 240,000 tokens moved from cached rate to base rate to save carrying a 400-token note.
This is why Manus's description of rewriting a todo.md as the agent works is safe rather than reckless: the rewrite is appended at the end of the context, which both preserves the prefix and keeps the current goal in the most recently attended position. The cache mechanics underneath are KV-cache budgeting for agent loops. The rule reduces to one sentence: files may be read into the tail of the context and must never be written into its head.
Step 5A file is state outside the trace, and three things you relied on quietly stop being true.
Once part of the agent's working memory lives on a disk, the transcript is no longer a complete record of the run. That has consequences beyond cost, and they are the ones that produce incidents rather than invoices.
- Replay stops being replay. Re-running a recorded trajectory reproduces the model calls but not the directory they read from, so a run that passed in CI can fail in production for reasons no trace shows. If you offload, your recorded artefact has to include a snapshot or a content hash of every file read — otherwise replay testing with recorded traces is testing a different program.
- Concurrency arrives without being invited. A shared scratch directory is a shared mutable store, so two sub-agents told to "record findings in
findings.md" will overwrite each other with no error and no conflict marker. This is precisely the failure catalogued in parallel writes and silent merges; the cheap fix is one file per writer plus a reducer, and the thing to decide deliberately is what a subagent inherits. - The file is an injection surface, including from yourself. Content the agent wrote at step 4 is read back as ordinary context at step 14, with no marker saying it was never trusted. If step 4's content came from a web page or an email, the instruction embedded in it has now been laundered through your own filesystem and arrives looking like the agent's own notes. Keep provenance on the written record — see context taint tracking and memory poisoning defenses.
The last one is the most commonly missed, because the file feels like infrastructure rather than input. It is input. A scratch directory an agent can write and later read is a persistence layer for prompt injection with a lifetime measured in sessions rather than turns, and the only general control is to label the origin of each written artefact and refuse to treat retrieved file content as instruction regardless of who wrote it.
Step 6Instrument the two ratios, and the rest of the decisions make themselves.
Almost nobody logs the numbers that decide whether offloading worked, which is why most teams cannot tell you whether it did. Two counters are enough, and both are cheap:
- Re-reads per file per run. A file read once is a correct offload. A file read on fourteen of twenty turns is a transcript with extra steps — either inline it, or summarise it into the pointer so the agent stops checking. The distribution is bimodal in practice, and the tail is where the money is.
- Bytes read ÷ bytes written. This is the selectivity ratio, and it is the single best predictor of whether the pattern is paying. Below about 0.1 you are getting the benefit. Approaching 1.0 you are paying inline cost plus round trips, and the fix is a better addressing primitive, not a bigger context window.
Set a policy threshold rather than deciding per call site: anything over a few thousand tokens goes to a file by default, with a generated pointer; anything under stays inline, because the pointer plus the round trip costs more than the content. Then make the read path narrow — offset and limit, search, or a code query — and treat a full-file read as a signal worth alerting on, the way you would treat a SELECT * in a hot path.
Do these three in order. First, add the offset/limit and search arguments to your file-read tool before you offload anything, because without them offloading cannot pay. Second, generate pointers automatically at write time — name, token count, shape, one addressing example — rather than letting the model invent a description of a file it has not read. Third, log re-reads per file and the bytes-read-to-bytes-written ratio for one week and look at the p95; if the ratio is near 1, you have not moved the content out of the loop, you have only moved it out of view. And whatever you do, do not splice a mutable notes file into the system prompt — that single move can cost more in lost cache hits than the entire offload saves.