Every one of these four frameworks will hand you a validated Python object instead of a string, and that is not where they differ. They differ on the one line of behaviour nobody reads the docs for: what happens when the model's answer does not fit the type. One retries on a budget, one raises into your request handler, and one retries or does not depending on which of three ways you declared the schema — where the most ergonomic declaration is the one that does not retry. That default is the whole decision, because it chooses whether your bad-output tail costs you a second inference call, a stack trace, or a wrong answer delivered with a straight face.
At a glance
The declaration syntax is a day-one concern and it is all anyone compares. The failure semantics are a month-three concern and they are what you will actually live with.
| Framework | How you declare it | On invalid output | Semantic rejection |
|---|---|---|---|
| Pydantic AI | output_type on the agent |
Feeds the error back, retries on a shared budget | ModelRetry from an output validator |
| OpenAI Agents SDK | output_type / outputType |
Raises ModelBehaviorError by default |
Opt-in run error handler returning a fallback |
LangChain create_agent |
response_format, three forms |
Retry handler applies to ToolStrategy only |
Not built in |
| Claude Agent SDK | JSON Schema, Zod or Pydantic | Leans on provider enforcement | Not built in |
Pydantic AI: the validation error is a prompt
Pydantic AI treats a failed validation as information to give the model rather than an error to report to you. A structured output that does not validate — and equally a tool call whose parameters do not validate — is passed back with a request to try again, which is the behaviour most people assume they are getting from all four and are getting from one.
The retry budget is shared and finite, and exhausting it raises UnexpectedModelBehavior with a message naming the limit: Exceeded maximum output retries. That is the right shape for a loop that could otherwise run forever, but note the budget is shared with tool-call retries, so a run that spent its allowance arguing about a tool signature has none left for the final answer.
The feature worth stealing is ModelRetry, which an output validator can raise to reject a result that passed the schema. This is the only first-class hook among the four for the failure that matters most in practice — the object is well-formed and the content is wrong. A schema cannot check that an ID resolves or a total adds up, and here you get to send that judgement back to the model in the same loop rather than bolting a second pass onto the outside.
OpenAI Agents SDK: raises, and that is defensible
The Python SDK's default when a final message fails to validate against output_type is to raise ModelBehaviorError. There is an opt-in run error handler for the invalid_final_output case which can return an application-specific fallback — and usefully, the SDK validates that fallback against the same output_type, so you cannot smuggle an untyped object back into a typed pipeline.
Raising is a real position, not an omission. It makes the failure loud at the boundary you control, which is exactly what the fail-closed argument asks for, and it refuses to spend your money retrying without being told to. The cost is that an unhandled ModelBehaviorError in a request path is an exception in production, and the JavaScript SDK has produced a steady trickle of reports from people who discovered this when a service went down — including one open issue, filed within days of writing against @openai/agents-core 0.21.0, where commentary and a final-answer JSON message arrive concatenated and validation fails on the pair.
Treat the handler as mandatory rather than optional. The default is correct and it is also not a production configuration.
LangChain create_agent: the ergonomic path is the one that gives up
This is the one worth reading carefully, because the trap is in the convenience. create_agent's response_format accepts three things: a ToolStrategy, a ProviderStrategy, or a bare schema type. Passing a bare Pydantic model — the obvious, documented, one-line thing to do — auto-selects ProviderStrategy.
The retry handler applies to ToolStrategy. So the bare-schema path, which is what you will write first and what most examples show, does not retry a malformed reply; the fix is to wrap the same schema in ToolStrategy explicitly. That asymmetry is not announced by the type signature, both forms produce a validated object on the happy path, and the difference only shows up in the tail.
There is a real trade underneath, which is why it is not simply a bug: provider-enforced structured output is cheaper and stricter, and a retry costs an extra model call. But the two options are presented as interchangeable ways to say the same thing when they have different reliability characteristics, and the shorter one is the weaker one. If you use create_agent, make the strategy explicit in your code even when the default would pick the one you want — the next reader cannot tell which you meant.
Claude Agent SDK: the failure moved upstream
The Claude Agent SDK accepts a JSON Schema, a Zod schema or a Pydantic model and returns a structured_output field carrying validated data. It has the least framework-level retry machinery of the four, and in a provider-enforced design that is close to consistent rather than missing: when the sampler is constrained against your schema, "the output did not fit the type" is not the failure you get.
What you get instead are the three failures that live outside the schema — a refusal, which arrives in a shape your schema never described; a truncation at the token cap, which yields a valid prefix of an invalid document; and a pre-generation rejection of the schema itself, which is deterministic and must never enter a retry loop. Anthropic's documented strict-mode limits are where the third one comes from: as of October 2026, at most 20 strict tools per request, 24 optional parameters in total across every strict schema in that request, 16 parameters carrying union types, and a 180-second schema compilation timeout, with an unsupported feature returning a 400 rather than degrading quietly.
So the comparison is not "this one has no retries." It is that the framework-level retry other SDKs give you is aimed at a failure this design mostly eliminates, while leaving three it cannot touch. Those are your job either way; structured outputs works through the four-branch parse path they imply.
Cross-cutting: a retry is an inference call, and a raise is a decision you deferred
Read across the four and there are only three places a non-conforming answer can go, and each framework's default picks one for you. Feeding the error back buys the best outcome and bills you for a second call on every occurrence, which is fine at a 1% failure rate and a budget line item at 15%. Raising spends nothing and converts a model problem into an availability problem, which is the right trade only if someone wrote the handler. Handing the object onward unchanged costs nothing and is the only one of the three that can produce a confidently wrong answer in front of a user, which is why it is the worst default to inherit by accident — and it is precisely the one you inherit by accident, in LangChain's bare-schema path.
The second thing that reads across is that none of the four frameworks, bar Pydantic AI's ModelRetry, has an opinion about the output being wrong. Schema conformance is the only test any of them applies by default, and conformance is necessary and never sufficient: numeric bounds and string-length keywords are not enforced by the strict modes underneath, so a validated object can carry an out-of-range number that your type annotation claims is impossible. Every one of these stacks needs semantic validation in your own code, and three of the four give you nowhere natural to put it.
Which leads to the thing worth measuring before you choose. Instrument your current stack for a week and get the rate: how often does a final answer fail validation? Below a percent or so, the raise-by-default position is fine and the retry is a rounding error either way. Above a few percent, the retry pays for itself in user-visible success and you should know what it is costing — and you should also go and look at the schema, because a double-digit invalid rate is usually a schema that asks the model questions it cannot answer rather than a framework that retries too rarely.
When to pick which
| Situation | Pick | Because |
|---|---|---|
| Extraction pipeline, batch, correctness over cost | Pydantic AI | Retries by default and ModelRetry lets you reject content that parsed but is wrong. |
| Synchronous request path with a strict latency budget | OpenAI Agents SDK | Raises instead of silently doubling your p99 — write the error handler and you have an explicit policy. |
| Already on LangGraph for durable execution | LangChain create_agent |
Fine, but pass ToolStrategy explicitly; the bare schema will not retry. |
| Claude-only, schema is stable and small | Claude Agent SDK | Provider enforcement removes the malformed-output case; budget for refusal, truncation and the schema 400 instead. |
| Schema has 20+ fields or deep optionality | None of the above, yet | Fix the schema first — strict modes ration or ban optional fields, so every field is a compulsory question. |
FAQ
Which of these four retries a bad structured output by default?
Pydantic AI does, feeding the validation error back to the model within a shared retry budget. LangChain's create_agent retries only under ToolStrategy, and a bare Pydantic model auto-selects the provider path that does not. The OpenAI Agents SDK raises ModelBehaviorError by default, and the Claude Agent SDK relies on provider-side enforcement instead.
Why does a bare Pydantic model behave differently from ToolStrategy in LangChain?
A bare schema auto-selects ProviderStrategy, which uses the provider's own structured-output enforcement, while the framework's retry handler is wired to ToolStrategy. Both return a validated object when things go well, so the difference is invisible until a reply is malformed. Pass the strategy explicitly.
Is raising an exception the wrong default?
No — it is a defensible one. It makes the failure loud at a boundary you control and refuses to spend tokens retrying without being asked. It is just not a production configuration on its own, so treat the opt-in error handler as mandatory rather than optional.
Does constrained decoding make framework retries unnecessary?
It removes the malformed-JSON case and leaves three it cannot touch: a refusal that arrives outside your schema, a truncation at the token cap that yields a valid prefix of an invalid document, and a deterministic pre-generation rejection of the schema. The last must never go into a retry loop.
Will any of these catch an answer that is well-formed but wrong?
Only Pydantic AI, via ModelRetry raised from an output validator. The other three apply schema conformance and nothing else, and conformance does not check that an ID resolves or a total adds up — numeric and string-length keywords are not enforced by the strict modes underneath. Semantic validation is your code's job in all four.
How do I decide without rewriting my stack?
Measure the rate at which final answers fail validation on your current setup. Under about one percent, the default barely matters. Above a few percent, look at the schema before you look at the framework — that rate usually means the schema is asking the model questions it cannot answer from the input.
Further reading
On this wiki:
- Structured outputs — what schema enforcement guarantees, and the four-branch parse path it leaves you.
- Constrained decoding — how the token mask is actually built.
- JSON Schema subsets per vendor — which keywords survive which provider, and what ports.
- Structured outputs vs tool calls — the same machinery, two consent surfaces.
- Agent frameworks — choosing on the interpose surface rather than the abstraction.