The Agent Harness

Model, framework, harness — three different things.

The words get used interchangeably and the distinction is load-bearing. A model is weights behind an API: it takes messages and returns text or a tool call. A framework is a library you build with — LangGraph, an agent SDK, whatever you imported. A harness is the specific assembly you actually run: this system prompt, these nineteen tools with these descriptions, this eviction policy, this stop condition, this retry behaviour, this thinking budget.

You can build a harness with a framework or with a while loop, and you can run two completely different harnesses on the same framework. The test that separates them: swap the model and keep everything else the same — everything else is the harness. Concretely it makes seven decisions the model never gets to make:

• What tools exist, at what granularity, described in what words. The description is prompt text you wrote, and tool choice moves with it.
• What enters context and what leaves it — truncation at the tool boundary, summarisation of old turns, whether a 50,000-token file goes in by value or by reference.
• When the loop stops — step ceiling, token ceiling, a satisfied-goal check, or nothing at all, which is the common case.
• What happens on a tool error — whether the model sees a usable message, a stack trace, or a retry it never learns about.
• What the agent may do unasked — which calls are auto-approved, which require a human, which are refused outright.
• How output is parsed — strict schema, lenient regex, or a repair loop that quietly changes what "the model said".
• How much thinking to buy — the effort or reasoning budget per step, which is a quality dial and a cost dial at once.

None of those live in the weights. All of them change the outcome.

Half the benchmark score belongs to the harness.

When a lab publishes an agentic result, it publishes a pair and names one element. Google's own evaluation notes for Gemini 3.7 Flash, released 13 August 2026, record that the coding results were self-computed using a mini SWE-agent harness with high thinking — the scaffold and the effort setting are part of the measurement, not incidental to it. That is standard practice, not a lapse; there is no way to score an agentic task without choosing a harness.

The size of the effect is easy to underrate. The same Gemini 3.7 Flash release reports roughly 85.8% on Terminal-Bench 2.1 and about 14.9% on Terminal-Bench 3.0. Identical weights, same week, one harder harness-and-task suite: a number that would look like a generational leap or collapse if you saw either in isolation. Two practical rules follow:

• A model comparison run on someone else's harness is evidence about their harness. It tells you which model wins under their tool catalog, their context policy and their effort budget. Under yours the ordering can and does invert — most often because their harness gives the model far more room to retry than yours does.
• Version the eval, not just the model. A score is only comparable to another score from the same suite at the same version with the same scaffold. Comparing across benchmark versions is the most common way a genuine regression gets shipped as an upgrade. This is the whole argument of reading benchmarks, sharpened by the fact that agentic suites carry a scaffold along with the tasks.

The harness is where your quality lives, and it is the cheap half to change.

Swapping models is a re-qualification: new refusal profile, new tool dialect, new context ceiling, a full eval pass before it is safe. Changing a tool description is a deploy. Given that asymmetry, harness work has by far the better return per unit of risk, and yet it is usually the last thing teams touch — because "upgrade the model" is a decision you can make in a meeting and "our tool errors are unreadable" is not.

The levers, roughly in order of how often they turn out to be the problem:

• Tool granularity. Forty flat, overlapping tools produce selection errors no model fixes. Fewer, better-separated tools with distinct names beat a bigger model reliably — see tool design for agents.
• Tool error messages. The error is a prompt. 400 Bad Request teaches nothing; "missing required field customer_id; call search_customers first" ends the failure in one step instead of six.
• Context policy. What you carry, you pay for on every subsequent step and the model attends to less of it. Context engineering is a harness concern end to end.
• Stop conditions. A missing termination rule is not a cost bug, it is a correctness bug: an agent with no stop rule eventually produces confident output about work it did not finish. See planning and termination.
• Effort budget. More thinking is a real quality lever with a real bill; it belongs under explicit control rather than left at whatever the vendor defaults to.

The uncomfortable version of this: a large share of "we tried agents and they didn't work" is a frontier model dropped into a harness with a sprawling tool catalog, raw API errors fed straight back to the model, and no ceiling on steps. The model was never the binding constraint.

Treat it as a versioned artifact, because that is what it is.

If the harness determines behaviour, then behaviour changes when the harness changes — and harnesses drift constantly, since a tool description edited by another team is a prompt change nobody logged. Three habits make that visible:

• Name the tuple. Behaviour is (model, prompt, tools, policy). Pin all four to versions, stamp the resulting hash on every trace, and you can answer "why did last Tuesday behave differently" — the discipline in rollout and versioning.
• Change one element at a time. A model upgrade shipped alongside a prompt rewrite produces an eval delta nobody can attribute, and both get blamed.
• Evaluate the assembly, not the model. Your eval set runs against your harness; a vendor's score against theirs is a hypothesis you have not tested. See agent evaluation.

This is also the clearest way to read the 2026 platform market. Managed agent runtimes are, precisely, vendor-supplied harnesses: you buy a loop, a tool-invocation path, a context policy and a set of safety controls that someone else tuned and maintains. That is a genuine product, and the trade is exactly the one this page describes — you stop owning the half of the system where most of your quality delta lives, in exchange for not having to build it.

Related: the agent loop for the mechanism the harness wraps, agent frameworks for the library layer underneath it, agent cost control for what the context policy does to the bill, and the benchmark landscape for which agentic suites are worth reading at all.