A benchmark where the best model in the world finishes 12% of the work is not telling you who to buy. It is telling you that nobody can do this job yet — and the only column on SemiAnalysis's new GDPx leaderboard with enough spread to change a decision is the one measuring dollars, where dividing cost by completion rate puts second place at roughly a third the price of first.
What GDPx is
SemiAnalysis published GDPx on 2 November 2026, credited to Max Kan, Andrew Megalaa and Dylan Patel. It is an attempt at measuring long-horizon professional work rather than exam questions: 206 tasks across 11 professional domains, assembled with 14 data partners, with individual tasks running up to eight hours and each one scored as the average of three attempts.
| Model | Score | Cost per task | Note |
|---|---|---|---|
| Claude Opus 5.5 | 12.0% | $51.41 | First overall; ~548,000 output tokens per task. |
| Gemini 4 Carbon | 10.9% | $16.29 | Second; reported as not yet generally available. |
| GPT-6 Astra | 9.6% | $53.99 | Third, and the most expensive of the three. |
| GLM 5.3 | 2.7% | — | Best-placed open-weight entry in the published set. |
One provenance note. The figures here come from the published leaderboard as reported across a wide set of independent write-ups in the two weeks after release, not from reading SemiAnalysis's methodology post directly — they are consistent across sources, which is the most that can honestly be claimed. Treat the exact decimals as quotable and the per-domain detail as approximate.
A 2.4-point spread at a 12% ceiling is not a ranking
Start with what the top of the table can and cannot support. First to third is 12.0% to 9.6%. On 206 tasks, a 2.4-point difference is roughly five tasks' worth of outcome, and each task's score is already an average over three attempts because single attempts were too noisy to use. The arithmetic of eval variance is unkind to gaps that size: at this set size and this success rate, the interval around each score comfortably overlaps its neighbours.
So "Opus 5.5 is the best agent model" is a claim the leaderboard does not quite make, and the authors' own averaging choice is the evidence. What it does establish, robustly, is the ceiling. Every closed frontier model is under 13%, and the best open-weight entry is at 2.7%. That is a statement about the task class, and it is the useful one — because a 12% ceiling says something no ranking can: for these jobs, the unattended agent is not the product yet. The deployable number is the one in task horizon, not the headline.
There is a reading of a low ceiling that is worth resisting, though. A new benchmark scores low partly because it is new — nothing has been trained against it, nothing has leaked into a training set, and the tasks were built with partners rather than scraped. The honest comparison is not "12% here versus 70% on a coding benchmark"; it is "12% on uncontaminated eight-hour professional work", and the right companion reading is benchmark contamination, which is the reason familiar numbers look better.
The cost column is the one with real spread, and it inverts the ranking
Published alongside the scores are average costs per task, and these have the spread the scores lack: $16.29 for Gemini 4 Carbon against $51.41 for Opus 5.5 and $53.99 for GPT-6 Astra. That is a 3.3× range on cost against a 2.4-point range on quality, which already tells you which axis your decision lives on.
It gets sharper when you do the division the leaderboard does not. The dollar figures are averages across all attempts, successful and failed — so dividing average cost by completion rate gives dollars per unit of work actually completed:
- Gemini 4 Carbon — $16.29 ÷ 0.109 ≈ $149
- Claude Opus 5.5 — $51.41 ÷ 0.120 ≈ $428
- GPT-6 Astra — $53.99 ÷ 0.096 ≈ $562
The quality leader is about 2.9× the cost of second place per unit of delivered work, and third place is 3.8× second. The ranking inverted, and it inverted using only numbers the leaderboard published.
Two honest caveats on that arithmetic, because it is the kind of calculation that gets screenshotted. It assumes a completed task is worth the same whichever model produced it, which is a real assumption on work this complex — a 12% that lands on the hard tasks is not interchangeable with a 10.9% that lands on easy ones. And a benchmark average is not a quote: your workload is not GDPx, Carbon was reported as not yet generally available, and list prices move. What survives both caveats is the shape of the thing. When quality differences are inside the noise and cost differences are multiples, the decision is a cost decision, and the correct way to run the comparison is at matched budget rather than at matched model — which is the whole argument of budget-matched evaluation, and the thing GDPx makes unusually easy to do because it published the dollars.
Two domains scored zero, and that is a different kind of result
Under the intended binary grading, finance and legal tasks produced zero for every model in the set, which is why SemiAnalysis reported average partial scores for those domains instead. The domains where there is a winner have different winners — quant trading, hardware and chip design to Opus; biology and, narrowly, software engineering to Carbon; recursive self-improvement and materials science to Astra.
That pattern matters more than the overall column. A domain with a universal zero is not reporting a capability gap between vendors; it is reporting that the task definition and the models are not yet in contact at all, and no amount of model selection changes your answer there. A domain with a clear winner is a real signal and it is per-domain — which means the overall average, the number everyone quotes, is a blend across eleven professions of which at most one or two resemble your work. This is the standard failure of leaderboard reading set out in reading agent benchmarks: the aggregate is the least informative cell in the table and the only one that fits in a headline.
The partial-credit substitution deserves one note of caution too. Swapping binary scoring for partial credit in exactly the domains where binary scoring returned zero is a reasonable editorial choice for readability, and it makes those two domains' numbers non-comparable with the other nine. Do not average across them.
Refusals score as failures, so the leaderboard partly ranks refusal policy
SemiAnalysis notes that safety classifiers depress some models' scores, particularly in cyber, biology and alignment tasks. Mechanically that is unavoidable: if a model declines a task, the task is not completed, and an automated scorer records a zero identical to the zero a wrong answer earns.
The consequence is that some unknown part of the gap between two models on those domains is a difference in deployment policy rather than in capability. That is not a complaint about the benchmark — a buyer who cannot get the task done does not care why — but it does change what the number predicts. Refusal behaviour is the most volatile property of a deployed model, re-tuned between minor versions, and it differs between the API and the consumer product; refusals and capability gating is the longer argument. A capability ranking is supposed to be stable for the life of a model version. A ranking that contains refusal rate is only as stable as the vendor's current policy.
The practical instruction: on any domain with a safety overlap, ask for refusals to be reported separately from failures before treating the score as capability. If the harness cannot distinguish them, that is a property of the harness — and as the agent harness argues, every agentic benchmark number scores a model and the scaffold it ran inside, and only one of them gets named.
What to take from it
Three things, and the first is the one most likely to be skipped.
Use the ceiling, not the ranking. The transferable finding is that uncontaminated multi-hour professional work is at roughly one task in eight for the best available model. That is an argument for scoping agents to steps a human reviews, and for measuring your own horizon on your own tasks rather than inheriting one.
Do the division. Any leaderboard that publishes cost alongside score lets you compute dollars per unit of completed work, and that number ranks differently from the score roughly whenever the score spread is small. It is one line of arithmetic and it is the number your finance team is going to ask for anyway — the framing in agent unit economics, where cost per successful task is only as trustworthy as the verifier deciding success.
Read the domain column before the total. Find the one or two of the eleven domains that resemble your workload, and ignore the average. If your domain is one of the two that scored zero, the benchmark's answer for you is that this is not a model-selection problem yet.
FAQ
What is the GDPx benchmark?
A long-horizon agentic benchmark published by SemiAnalysis on 2 November 2026: 206 tasks across 11 professional domains, built with 14 data partners, with tasks running up to eight hours and each scored as the average of three attempts.
Which model leads GDPx?
Claude Opus 5.5 at 12.0%, followed by Gemini 4 Carbon at 10.9% and GPT-6 Astra at 9.6%. The best-placed open-weight entry in the published set is GLM 5.3 at 2.7%.
Why is the top score only 12%?
Because the tasks are multi-hour professional work rather than exam questions, and because the benchmark is new, so nothing has been trained or tuned against it. A low score on a fresh, uncontaminated set is not directly comparable to a high score on a well-known one.
Is the 2.4-point gap between first and third meaningful?
Not on its own. Across 206 tasks at these success rates the intervals around each score overlap, which is also why each task is scored as an average of three attempts. The ceiling is the robust finding; the ordering at the top is not.
Which model is the best value on GDPx?
Gemini 4 Carbon, by a wide margin, if you divide published cost by completion rate: about $149 per unit of completed work against $428 for Opus 5.5 and $562 for GPT-6 Astra. That assumes a completed task is worth the same whichever model produced it, and benchmark averages are not production quotes.
Do refusals count against a model's score?
Yes — a declined task is an incomplete task, and an automated scorer cannot tell it from a wrong answer. SemiAnalysis notes that safety classifiers depress some scores, particularly in cyber, biology and alignment, so part of the ranking on those domains reflects policy rather than capability.
Further reading
On this wiki:
- Budget-matched evaluation — why comparing models at equal spend, not equal name, is the comparison you want.
- Reading agent benchmarks — why the aggregate is the least informative cell in the table.
- Eval variance & statistical power — the interval around a score, and why small gaps are not results.
- Task horizon — the capability number stated in a unit you can plan with, and why the headline figure is not the deployable one.
- The agent harness — every agentic score grades a model and its scaffold, and only one of them is named.
- Refusals & capability gating — why refusal rate is the most volatile property of a deployed model.