Transparency & Explainability

8 min read

H3
Playbook · Agent UX & Human Interaction

You can no longer show the reasoning, because the API stopped giving you one.

The design argument about whether to display an agent's chain of thought was settled in 2026 by the vendors, against you: as of October 2026 every major reasoning API returns its hidden reasoning as an encrypted payload you must pass back unread, plus an optional human-readable summary produced by a separate pass. So the thing your UI can show is no longer a trace — it is a vendor's paraphrase of a trace, two unverifiable hops from the computation. That makes the checkable tier of transparency the only tier you control, and the good news is that your own harness generates all of it: the exact tool call, the query that ran, the diff that will apply, the passage that was quoted.

Step 1Start from what the API actually returns, because it is not what the design conversation assumes.

Most transparency UX writing still assumes you have a reasoning string and a choice about whether to render it. Check the current provider documentation before designing around that assumption. As of October 2026:

  • Anthropic returns thinking blocks whose signature field carries the reasoning content encrypted, with the readable part governed by a mode — summarized is documented as the default on Claude Opus 4.6, Claude Sonnet 4.6 and earlier models, and an omitted mode returns thinking blocks whose thinking field is empty. The docs instruct you to pass the signature back unchanged on multi-turn and tool-use conversations.
  • Google returns an encrypted thought signature described as a representation of the model's internal reasoning, required to maintain reasoning continuity across turns; in stateless use the documentation says you must resend all thought blocks exactly as received. Thought summaries may come back empty depending on configuration, how much reasoning the model did, and the content type.
  • OpenAI returns the reasoning item encrypted, surfaced through a reasoning.encrypted_content field.

Three consequences land at once, and only the first is widely noticed. You cannot render the trace. The readable summary is a different artifact, generated by a separate process, whose fidelity to the hidden reasoning nobody has published a measurement of. And the opaque blob is now a required part of your request payload, which makes it an operational object with handling rules — step 4.

Do not label a provider summary "the agent's reasoning" in your UI. It is a summary of a thing you cannot see, and the label makes a causal claim on the user's behalf that neither you nor the vendor has evidence for. If you render it, call it what it is: a summary the model wrote about its own thinking.

Step 2There are now three gaps between the explanation and the computation, not one.

The original problem was already two-sided. Plausibility is whether an explanation reads as sensible; faithfulness is whether it reflects the computation that produced the output. Work such as Barez et al., Chain-of-Thought Is Not Explainability (Oxford, 2025) and the broader faithfulness literature shows these come apart sharply, and models reliably produce plausible rationales while the real cause was a prompt cue they never mention. Users cannot perceive faithfulness at all — a fluent rationale feels faithful — so a persuasive-but-unfaithful explanation is an over-trust generator. That remains the core risk, and chain-of-thought faithfulness is the concept page for it.

The 2026 API shape inserts a third gap in front of the other two:

  • Gap 1 — summary to trace. The summary is generated from the hidden reasoning by another pass. Compression is lossy and no provider publishes a fidelity number.
  • Gap 2 — trace to computation. The pre-2026 faithfulness problem, undiminished. The hidden trace was never a causal account either.
  • Gap 3 — computation to behaviour. The tokens the model emitted are not the whole of what determined the tool call it made.

Chaining three unverified mappings does not produce a weak explanation; it produces a narrative with no established relationship to anything. Treat it as a hypothesis-forming aid for your own debugging and never as evidence a reviewer should weigh. If you genuinely need to know why a model did something, the honest answers are behavioural and mechanistic, not introspective — see interpretability and probes, and note that the counterfactual probing in evaluating an agent you can only call is the same technique pointed at your own stack.

Step 3Build the checkable tier yourself — it is the only part no vendor can withdraw.

Rank explanation types by how independently checkable they are, and notice who produces each one:

  • Strongest — externally grounded, and generated by your harness. The exact tool call and its raw result, the SQL that ran, the diff that will be applied, the retrieved passage with its document id and timestamp. These are facts about the world a user can inspect without trusting anyone's introspection, and critically they come from your code. No API change can take them away.
  • Middle — structural, and partly yours. The plan, the decomposition, which tools were chosen and which were skipped. You can reconstruct most of this from the call sequence even when the reasoning is sealed; coherent, but not evidence of hidden causes.
  • Weakest — introspective, and now a vendor's paraphrase. The free-text account of why. Useful for hypothesis-forming, never as evidence of correctness, and in 2026 further removed from the computation than it was.

The practical shift is that the top tier has to be engineered deliberately rather than harvested. A harness that logs tool_use blocks with full arguments, results and timings has a trajectory; one that logs "called search" has nothing to show. That is the same instrumentation trajectories and decision receipts describe, and the transparency UI is simply its most user-facing consumer.

What this looks like in a real product — a support agent issuing a refund, in descending order of what a reviewer can actually check:

# Tier 1 — your harness produced every field. Checkable without trust.
refund.create(order="A-4471", amount_cents=2300, reason="damaged")
  policy: refunds.md#damaged-goods (rev 2026-09-14)
  evidence: ticket 88213, photo attachment 2, received 2026-10-17

# Tier 2 — reconstructed from the call sequence, not from self-report.
steps: lookup_order -> fetch_policy -> check_window(14d, ok) -> refund.create
skipped: escalate_to_human (threshold $100, not met)

# Tier 3 — a summary the model wrote about its own hidden reasoning.
# Label it as such. Never render it as "why the agent did this".

A reviewer can verify every line of tier 1 against the order, the policy file and the photo. Tier 2 they can check for coherence. Tier 3 they can only believe — which is why it sits below the fold, if it appears at all. Attribution of the sources themselves is its own craft; see citations and source attribution UX.

Step 4Treat the reasoning signature as a credential, not as content.

The encrypted block is a new object in your system with properties no transparency design has historically had to think about, and two of them are security properties.

  • It must round-trip byte-exact. Both Anthropic and Google document that the signature or thought block is passed back unmodified to preserve reasoning continuity across turns and tool calls. A middleware that normalises, truncates or strips unknown fields will break multi-turn reasoning in a way that looks like a model regression — the failure mode carrying reasoning across tool calls exists to describe.
  • It is not safe to echo. An August 2026 arXiv paper, Stealing Reasoning Traces from Proprietary LLM APIs, reports that an encrypted reasoning payload can be inserted into a compatible model's context with a prompt asking it to repeat the prior thinking verbatim, recovering plaintext. Take the specific result as reported rather than confirmed — but the design implication needs no confirmation: a debug panel, a support export or a verbose log that includes the blob is a channel for whatever the hidden reasoning contained about your prompt and your user's data. Keep it out of user-visible surfaces and out of anything you hand to a third party, and apply the same handling as the rest of a trace under PII redaction in agent traces.
  • You are paying for it. Thinking tokens bill at output rates on the major providers, so "more transparency" in this sense is a line item rather than a free display toggle — the accounting is in reasoning token accounting.

Concretely: store the signature, never render it, never put it in a URL or a support ticket, and give it the shortest retention your continuity needs allow. It behaves like a bearer token that happens to be derived from your prompt.

Step 5Explain at the altitude the decision requires, and lead with provenance.

There is no single right depth — there is the depth that supports this user's next decision. Over-explaining is its own failure: a wall of reasoning trains users to collapse the panel unread, and you lose the channel exactly as with over-displayed confidence in H1.

# Altitude follows consequence and the user's role
if stakes == "high" or user.is_reviewing:
    show(plan + sources + diff)        # inspectable substrate
elif user.asked_why:
    show(one_line_rationale, expandable=True)
else:
    show(outcome + provenance_badge)   # quiet by default

For most agent outputs the single most decision-useful piece of transparency is not why but from where: which source, which tool, which retrieved passage, what timestamp. Provenance is externally verifiable, hard to fabricate convincingly, and directly actionable — a user can click through and check. It also degrades gracefully: "I couldn't find a source for this" is itself honest, high-value transparency, and an agent that says it earns more calibrated trust than one that always has a fluent answer. That honesty has a structured form worth adopting — structured refusal and why-trails.

And transparency that only narrates the chosen path hides what a reviewer most needs: the close alternative the agent rejected, the assumption it had to make, the input it lacked. Surfacing those converts the explanation from a justification — which motivates rationalisation — into a decision aid, which invites correction. Where the user can act on it before execution, that is simply plan review; at scale it is the queue design in review queues for agent output.

Default to a one-line "what and from where", make the full plan, sources and diff one click away, and never auto-expand a reasoning summary. Depth on demand beats depth by default — and the one-line default is cheap to build because your harness already knows the tool and the source.

Step 6When NOT to show a reasoning trace.

You mostly cannot, so the rule is sharper than it was: never present a provider's reasoning summary as the agent's reasoning, and never let its presence substitute for the checkable substrate — surface the tool call, the source and the diff, and let any self-reported narrative stay an explicit, opt-in hypothesis.

Two cases where even the summary should be off. If the user's next action is to approve something irreversible, a fluent rationale measurably raises confidence without raising correctness, which is the worst possible combination at a gate. And if your product is audited, a summary in the record invites an auditor to treat it as a reason given — so keep the explanation field populated by your harness, not by the model.