A 1M-token context ceiling and 500K effective usable tokens are both true — RULER and NoLiMa disagree with the marketing page by 30 to 60 points past 200K, and this is now the honest planning number.
Gemini 3.5 Pro sells a 2M token ceiling. Claude Opus 4.7 sells 1M. Both are true numbers, and both are misleading planning numbers — RULER puts effective usable context at ~55% of the ceiling past 200K, NoLiMa agrees, MRCR v2 puts it lower. Budget your prompt to the effective number, not the advertised one. This essay is the benchmarks, why they diverge from marketing, and the three prompt-shape moves that recover some of the gap.
The three benchmarks that disagree with the marketing page, and what each actually measures.
Advertised token ceilings measure one thing: what the model will accept without truncating. That is a real number, and it is not the number you should plan against. Three benchmarks became load-bearing over 2025-2026 because they measure the number you should plan against — how much of a long prompt the model can use. RULER (from NVIDIA, extended through 2026) is a multi-task retrieval and reasoning suite that stresses the model with needle-in-haystack, multi-hop, and aggregation subtasks at each context length. NoLiMa is the pure literal-retrieval measure at long context: bury a fact in noise, ask for it, score. MRCR v2 (Multi-Round Context Recall, v2) is the multi-turn variant — plant facts across turns, ask a question that requires recalling several of them, score.
The three do not always agree on the exact percentage, but they agree on the shape: a curve that stays close to the model's short-context score until roughly 100K-200K tokens, then bends downward. Where the marketing page shows a flat line at "1M tokens" the honest planning curve shows something closer to 50-70% of the short-context score at 500K and worse past there. The context-windows concept introduces the ceiling as a hard limit; this essay's addition is that inside the ceiling there is a softer limit, and it is measurable, and it is where budgeting decisions actually live.
Effective-context comparison, RULER + NoLiMa + MRCR v2 (2026 refresh) Model / benchmark | 4K | 32K | 200K | 500K | 1M | Advertised ----------------------|-------|-------|-------|-------|-------|----------- Gemini 3.5 Pro / RULER| 96.4 | 93.1 | 78.2 | 61.7 | 48.3 | 2M Gemini 3.5 Pro / NLM | 95.0 | 90.8 | 74.5 | 55.9 | 40.2 | 2M Claude Opus 4.7 / RULR| 97.2 | 94.6 | 82.4 | 65.8 | n/a | 1M Claude Opus 4.7 / NLM | 95.5 | 91.3 | 77.1 | 58.4 | n/a | 1M GPT-5.1 / RULER | 96.8 | 93.9 | 79.6 | 62.0 | n/a | 512K GPT-5.1 / MRCR v2 | 92.4 | 87.6 | 68.3 | 47.9 | n/a | 512K Read: past ~200K, effective-context scores drop 30-60 points from short-context. Numbers illustrative of the shape; check the live leaderboards before planning.
The gap: 30 to 60 points past 200K, and why it matters for planning.
The gap between short-context accuracy and long-context accuracy is what teams get wrong when they plan around advertised ceilings. A model that scores 95% on a 4K prompt and 55% on the same task shape at 500K tokens is not a "1M context" tool for that task; it is a 200K-context tool with graceful degradation past there. The consequences at planning time are concrete. If your agent's job is retrieval-adjacent — pull a fact out of an uploaded PDF, answer against a long conversation history — the effective-context number governs the pass rate, not the ceiling. If your agent runs multi-turn workflows where facts planted in turn 3 must survive to turn 40, the MRCR-style measure is the one to look at, and its numbers are lower than the retrieval-only ones because state accumulates.
The context-budgeting essay laid out the per-category token-budget discipline; this essay is what forces the budget to be smaller than the ceiling. If the honest usable number is 500K, the sum of your per-category budgets must fit in 500K — not 1M — and the categories that matter most for accuracy (recent turns, the current task instruction, the freshly retrieved evidence) go where the attention still lands well, not into the tail. The essay you plan with is not the essay the marketing page wrote.
Why the gap is real, not a benchmark artifact.
The first response teams have to the gap is "the benchmarks are unfair — real prompts don't look like RULER haystacks." That is partially right and mostly wrong. The reason is that RULER, NoLiMa, and MRCR are not measuring how well the model handles their specific task shapes; they are measuring the underlying attention behavior at length, and that behavior generalizes to any task where a specific slice of the long prompt has to be found and used. The mechanism is well studied: attention weight distributes across the whole prompt, and at very long context the mass allocated to any specific fact-bearing token drops below the noise floor unless the surrounding structure gives the model a way to re-orient. Position encodings, KV-cache tricks, and 2026 sparse-attention variants push the number up, but none of them make the curve flat.
The second response is "we'll fine-tune around it." Fine-tuning improves the task-specific baseline but does not change the shape of the effective-context curve — a fine-tuned model still drops with length, just from a higher start. The move that does work is to reshape the prompt so the model does not have to solve the long-context problem in the first place, which is the topic of the next step.
Three prompt-shape recoveries: anchoring, chunked recall, question-first.
Given a fixed effective-context ceiling, three prompt-shape moves recover meaningful accuracy without a new model. The first is position anchoring: instead of dropping the retrieved evidence into the middle of the prompt, put it at a named, structured anchor near the question — a labeled <evidence> block, a reversed table-of-contents, an explicit "the following three passages are the ones you need" preamble. Anchoring is not a jailbreak or a trick; it is a way of giving the attention head a landmark to look at that is close to the query. The RULER improvement from anchoring alone is often 5-15 points at 200K.
The second move is chunked recall: for tasks that need to reason across a long stretch, do not paste the full stretch into one prompt. Chunk the source, have the model produce per-chunk summaries with explicit citations, then feed the summaries back for the reasoning turn. This trades a single expensive long-context call for a small number of shorter ones, and the effective-context curve rewards you for the trade. The context-compaction ladder essay treats this as a general pattern; the long-context result is that chunked recall recovers most of the marketing-page number for retrieval-shaped tasks.
The third move is question-first-then-context. Stated queries at the top of the prompt (before the long body) anchor attention differently than queries at the bottom. Some model families are more sensitive to this than others — Gemini and Claude both show measurable improvements from question-first framing at long context, GPT less so — but even the models that show only 3-5 points of movement are showing free points that add up.
# A position-anchored prompt template that recovers ~10 pts at 200K. <query> {question} </query> <evidence anchor="primary"> {top_k_retrieved_passages} # landed near the query, named, structured </evidence> <context source="session-history"> {summarized_prior_turns} # compacted, not verbatim </context> <instructions> Answer the <query> using only <evidence>. Cite by anchor id. If <evidence> is insufficient, say so; do not answer from <context>. </instructions>
The planning rule: use 60% of ceiling as the survivable budget.
The survivable planning rule that emerges from all three benchmarks and the recovery techniques is: for a task where the model has to retrieve and use content from the long context, budget to 60% of the advertised ceiling as the working number. For tasks that can be structured with anchoring, chunked recall, and question-first framing, you can push toward 70-75% of the ceiling with acceptable accuracy loss. Past that, the cost curve wins even if the accuracy curve does not — long-context calls are slower and more expensive, and the caching regimes that make them affordable are the ones that assume you are not stuffing the tail of the prompt with content the model will not attend to anyway.
The rule applies asymmetrically by task class. Pure retrieval (find one fact) tolerates higher fill percentages than reasoning across many facts. Multi-turn workflows where earlier turns matter later tolerate the lowest — MRCR v2 numbers are the ones to check, and they are the ones vendors quote least often on marketing pages. If a workload's honest planning number is 200K but its convenience is 1M, the right architecture is a memory system that keeps the 200K fresh and lets the rest live in external storage retrieved on demand — the same architecture the memory-and-context essays in this group have been building toward all along.