Credit Assignment: Which Agent Do You Change?

Name the problem correctly: this is credit assignment.

Reinforcement learning has a name for this and a century of pain around it. You observe a reward at the end of an episode and you must distribute responsibility across the decisions that produced it — temporal credit assignment across steps, and in a multi-agent setting structural credit assignment across actors. A supervisor decomposed the task, three workers executed, a critic reviewed, and the final answer was wrong. Which of the five decisions was the one to change?

RL solves this with volume and an algorithm: millions of episodes, a value function, a learned baseline. You have forty traces, a person reading them, and a deadline. That asymmetry is the whole reason the practice looks nothing like the theory, and it is why the honest methods here are estimators with known biases rather than solutions.

Two properties make the multi-agent case harder than single-agent trajectory analysis. First, the failure is frequently in the coupling rather than in any component — a worker that did exactly what it was asked, correctly, when the ask was wrong. Second, the components interact, so responsibility is not additive: fixing agent B alone can make the run worse if agent C had been silently compensating for B's output. Any method that produces a per-agent score is implicitly claiming additivity it cannot support.

Estimator one: per-agent judges. Cheap, parallel, and locally scoped.

Score each agent's output against its own brief, in isolation. A judge sees the sub-task it was given and what it produced, and rates it. This is the default because it composes with everything you already have: one judge configuration, run N times, no re-execution, results in seconds.

What it measures is local quality: conditional on the input this agent received, was its output good? That is a real and useful quantity. It catches a worker producing malformed output, a critic that rubber-stamps, a summariser that drops caveats.

What it cannot see is the seam. Consider the canonical coordination failure: the supervisor decomposes a research task into three sub-questions, none of which is the question the user asked. Every worker answers its sub-question well. Every per-agent judge returns a high score. The system's answer is useless, and your attribution says everything is fine — the method has systematically exonerated all five components of a failure that lives between them. This is the same blind spot that makes per-component health checks useless against error propagation, arriving through the evaluation door.

There is a second, subtler bias. Per-agent judges score conditional on the input received, so an agent given a corrupted input by an upstream agent is judged on how well it handled garbage. If it handled garbage gracefully, it scores well — and the run still failed. Local quality and system contribution can move in opposite directions, and the score gives you no way to tell which case you are in.

Use it for what it is: a screening step that finds obviously broken components fast. Do not let a bar chart of per-agent scores drive a decision about which agent to remove or rewrite.

Estimator two: ablation. Causal, expensive, and drowning in variance.

Remove an agent — or replace it with a trivial pass-through — re-run the suite, and measure the change in end-to-end success. This is a genuine intervention, and it answers a question worth asking: does this component contribute?

Two costs. The first is obvious: one full suite re-run per component, so an eight-agent system is eight suites, and if you want pairwise interactions it is worse than that. The second is the one that wrecks conclusions.

Agent runs are high-variance. The same configuration on the same input produces different trajectories, and end-to-end success rates on a suite of forty tasks carry a wide confidence interval. An ablation delta of four percentage points against a noise floor of eight tells you nothing at all, and it will be reported as "agent C contributes 4%" anyway. Before running ablations, do the power calculation — how many tasks and how many repetitions are needed for the smallest delta you would act on to clear the noise — exactly as laid out in eval variance and statistical power. Most teams discover the answer is several times the suite they have.

There is also an interpretive trap. Ablation measures marginal contribution in the current configuration, so an agent whose job is fully absorbed by a downstream agent's robustness will ablate to zero delta and look redundant. Sometimes that is a real finding and you should delete it. Sometimes you have just measured that another component is quietly doing double duty and paying for it in tokens — which is a different fact with a different fix.

# ablation is only informative once the delta clears the noise floor
delta = success_with - success_without
if abs(delta) < noise_floor(n_tasks, n_reps):
    report("inconclusive")  # not "no contribution"

Reserve ablation for structural decisions — should this agent exist, should this topology collapse to a single agent — where the answer justifies eight suite runs and where the delta you care about is large. It is the right tool for the question when to go multi-agent asks, and the wrong tool for debugging a specific failure.

Estimator three: counterfactual replay. The highest-information option, and the one nobody builds.

Take a recorded trace of a failed run. Replace exactly one agent's output with a known-good substitute — a human-written answer, an oracle result, or the output of a stronger model — and continue execution from that point with everything else held fixed. If the run now succeeds, that output caused the failure. If it still fails, it did not, and you look upstream.

This answers the third question directly: did this specific output cause this specific failure? It is the only one of the three estimators that does, and it is enormously more informative per run than ablation because it holds the entire rest of the trajectory constant instead of re-rolling it.

The reason it is rare is engineering, not theory. It requires a trace that stores every inter-agent message with enough fidelity to resume from any point, which means your tracing has to be a replay log rather than a debugging aid. It requires the environment to be resumable — trivial for a pure-reasoning pipeline, hard the moment agents have written to a database or called a paid API. And it requires a known-good substitute, which for many sub-tasks means a human in the loop, so the method does not scale to every failure.

Where it pays for itself is a recurring failure pattern. Once the same class of failure has appeared five times, building replay for that path costs less than the fifth argument about whose fault it is, and it settles the question with an experiment rather than a reading. It is also the technique that generalises into training data: a replay that isolates one bad step is exactly the supervision signal a process reward model wants.

Three questions, three tools, one table to keep on the wall.

• "Was this agent's output good?" → per-agent judges. Seconds, no re-execution, screens for broken components. Blind to coordination failures and to inherited garbage. Never causal.
• "Does this agent contribute?" → ablation. Causal, N suite re-runs, needs a power calculation before it means anything. Answers structural questions about the topology, not questions about a specific bug.
• "Did this output cause this failure?" → counterfactual replay. Causal and specific, needs replayable traces and a known-good substitute. Expensive to build once, cheap per use, worth it only for a recurring pattern.

The failure mode is running the cheapest one and reporting it as though you had run the most specific one. "Our attribution shows the retrieval agent is the weak link" almost always means per-agent judges scored it lowest, which is compatible with the retrieval agent being handed a bad query by a supervisor nobody measured — the exact situation in which changing the retrieval agent will not help and will consume a sprint.

Note what none of the three does: attribute across time within a single agent. All of them treat an agent as an atom. If your failure is a worker that was fine for nine steps and drifted on the tenth, you need step-level trajectory analysis instead, which is the subject trajectory and process evaluation takes up.

The protocol that actually works at forty traces.

At the scale most teams operate — dozens of traces, not millions — the highest-yield method is unglamorous and manual, and it should be step one every time.

1. Read twenty failed traces by hand and mark the first divergence. For each, find the earliest point where the system state stopped being on a path to the goal. Do not score anything; just record the step index and the actor. At N=20, a human reading traces beats every automated attribution method available, and it costs an afternoon. The distribution of first divergences is your actual priority list.
2. Cluster the divergences before fixing anything. Twelve of twenty landing on the supervisor's decomposition is a design finding, not a prompt-tuning finding. A flat distribution across all agents usually means the failure is in the protocol between them.
3. Fix the modal cluster, and re-run the same twenty. Same traces, same tasks — you are looking for the divergence point to move, which is a far more sensitive signal than an aggregate success rate at this sample size.
4. Escalate to an estimator only for the question you now have. If the question became "do we need this agent at all", run the ablation with enough repetitions. If it became "this one handoff keeps corrupting", build replay for that path. If it is neither, you did not need an estimator.
5. Only then automate. Encode the first-divergence judgement as a judge once you have a labelled set of twenty to calibrate it against — and hold it to the same standard as any other judge, with agreement measured against your labels rather than assumed.