Public-Benefits Casework Agents

A determination is a legal act. The agent may assemble; it may not decide.

Approving or denying a benefit is not a recommendation with a confidence score. It creates or extinguishes an entitlement, it must be accompanied by reasons, and it is appealable — which means every part of it has to survive being read back by a tribunal.

• Split the work at the only line that matters: gathering, verifying and organising evidence on one side; applying the rule to the facts on the other. The first half is where the weeks go and where an agent is genuinely excellent. The second half is the act, and it stays with a caseworker who can be named in the file.
• "A human reviews it" is not the same as a human deciding. If the agent produces a determination and the caseworker's job is to approve it, you have built MiDAS with a rubber stamp — the human is the throughput bottleneck the system was bought to remove, so the pressure is entirely toward approving faster. Have the agent produce the packet, not the verdict: the facts found, the evidence for each, the rules that apply, and what is missing.
• Adverse actions and favourable ones are not symmetric. A denial, a termination, a repayment demand or a fraud referral needs a different bar than an approval, because it takes something away and the burden lands on the claimant to reverse it. Where an agent may reasonably shorten the path to "yes", it should never shorten the path to "no".
• Never let the agent close a case. Auto-closure for non-response is where the harm concentrates: the claimant who missed a notice is disproportionately the one who most needed the benefit and least reliably received mail. Route non-response to a human queue, not to a terminal state.

Policy is versioned law, so retrieval must be pinned to the date of the claim.

This is the requirement that quietly breaks every standard retrieval stack, and it is the one nobody writes down. Eligibility is determined by the rules in force during the period claimed — not the rules in your index today.

• A claim filed today may cover a period governed by three different rule sets. Income thresholds change annually, categorical eligibility changes with legislation, and emergency provisions switch on and off with an effective date. A retrieval layer that returns "the current policy manual" produces a confidently wrong answer on every backdated claim, and the answer looks exactly like a correct one.
• Store an effective-from and effective-to on every policy chunk, and make the claim date a mandatory filter — not a ranking signal. If the date is missing, the retrieval must fail rather than fall back to current policy. This is the one place where a hard error is enormously better than a graceful degradation.
• The corpus is larger than the statute. Regulations, agency manuals, policy memoranda, waivers, and the appeal decisions that interpret all of them. Each has its own effective dating and its own precedence, and an agent that cites a manual paragraph superseded eighteen months ago has produced a defect that reads as authoritative.
• Cite to the version, not to the document. Every assertion in the packet points at a specific paragraph of a specific version, with a locator a human can open. Without that, nobody can check the agent's work in the time the queue allows, which is the whole argument in citations and source-attribution UX.

Freeze the decision packet, because the appeal arrives months later.

An appeal is heard long after the run, against a system that has since changed model, prompt, index and policy corpus. If you cannot reconstruct what the caseworker was shown, the agency's position at the hearing is unsupportable.

• Snapshot what was presented, not just what was stored. The retrieved passages, the rule versions applied, the facts asserted with their sources, the model and prompt version, and what the caseworker saw on screen at the moment of determination. Regenerating it later produces a different packet, and a different packet is not evidence.
• The packet is a record, not telemetry. It must sit outside any sampling or retention policy that could delete it — the distinction argued in trace sampling and retention — and its lifetime is the appeal window plus the statutory record-keeping period, not thirty days.
• Record what the agent could not verify. A packet listing four facts as confirmed and two as unverified is far more useful at a hearing than one that presents six with equal confidence. The unverified list is also the work queue for the caseworker.
• Keep the human's edits. When a caseworker overrides the agent's finding, that disagreement is simultaneously the strongest evidence of independent judgement and the highest-value training and evaluation signal you will get — see production feedback signals. Log the override and the reason as a first-class field, never as free text nobody reads.

The dangerous error is the denial nobody appeals, and appeal rate cannot detect it.

Every dashboard in this domain reports overturn rate on appeal, and it is a broken metric in the specific direction that matters, because appealing requires exactly the capacity that the affected population is least likely to have.

• Selection bias runs the wrong way. The claimant who appeals has time, literacy, a stable address, and often help. The claimant wrongly denied who does not appeal simply disappears from your data, and your quality metric improves. In both MiDAS and the Dutch case, the people harmed most were least equipped to contest it, and that is why the failures ran for years.
• Measure with a sampled audit, not with complaints. Pull a random sample of determinations — weighted toward denials — and have experienced adjudicators re-decide them blind to the original outcome. That number is your actual error rate; the appeal statistics are a measure of who could reach you.
• Watch the denial rate by segment as a leading indicator. Compare against the pre-agent baseline by claim type, language, channel and geography. A shift concentrated in one segment is the signal that arrives before the lawsuit, and it needs no ground truth to compute.
• Track time-to-determination and abandonment together. An agent that speeds up approvals while raising the share of claimants who abandon mid-process has moved the harm rather than reduced it — the requests for additional evidence it generates are a burden, and each one is a chance to lose someone.
• Never let the agent make a fraud referral. Fraud is an accusation with criminal exposure and it is where automated systems have caused the most documented damage. An agent may flag an inconsistency for review; the referral is a human act with a human name on it.

The notice is the product, and the reason given must be the actual reason.

The claimant does not experience the case packet; they experience a letter. If the letter does not say what specifically was missing or wrong, the appeal right is nominal — the person cannot fix a defect that was never named.

• The stated reason must be the operative one. A plausible summary generated after the fact is a post-hoc rationalisation, and it will diverge from the record under examination. Generate the notice from the same structured findings that drove the determination, so the two cannot disagree.
• Say what to do next, specifically. Which document, covering which period, sent where, by when. "Insufficient documentation" is a non-notice; it names a conclusion rather than an action, and it converts a fixable gap into a denial.
• Write for the reader, in their language, at a reading level you have actually tested. This is the one place where the technology is unambiguously good news: plain-language rewriting and faithful translation of a fixed set of findings is a well-shaped generation task with a checkable output, and it improves outcomes for exactly the population that appeal statistics were hiding. Keep the terms of art that carry legal meaning intact, per translation and localization agents.
• Disclose that an automated system was involved, and how to reach a human. Beyond being an obligation in most jurisdictions now, an undisclosed automated denial is the fact that turns an administrative error into a scandal. The mechanics are in disclosure and content provenance.

Where to start, and what will actually reduce the backlog.

The instinct is to automate adjudication because that is where the queue visibly sits. The return is higher, and the risk far lower, on everything that happens before a decision is possible.

• Start with intake completeness. Most of the backlog is not undecided cases, it is cases waiting on a missing document. An agent that reads a submission on arrival, tells the claimant exactly what is missing while they are still in the session, and re-checks on resubmission removes weeks of round-trip without touching a determination.
• Then evidence assembly. Pulling the wage records, the prior claims, the identity verification, the relevant policy versions, and laying them beside the elements of the test — this is the caseworker's slow work, it is retrieval rather than judgement, and it is fully auditable.
• Then triage by complexity, never by predicted outcome. Routing "likely denials" to a fast lane is how a prediction becomes a determination through the back door. Route by how much verification a case needs, which is a property of the evidence rather than of the answer.
• Treat submitted documents as untrusted input. Claimants upload PDFs and images, some of which will be adversarial. Isolate parsing, never let document content reach a tool-enabled context unfiltered, and apply prompt-injection containment as an architectural requirement rather than a policy.
• Procure with the record in mind. If the model, the policy corpus or the vendor changes, the agency still owes a reconstruction of decisions made under the old stack. Version pinning and an exportable decision record belong in the contract, not in a later migration plan — the vendor questions are in third-party model and vendor risk.

Related: insurance claims agents for the same evidence-assembly shape under a private-law duty, KYC & AML onboarding agents for the false-positive backlog pattern, human-in-the-loop for keeping the review real, and the EU AI Act for agents for why access to public benefits is treated as high-risk.