Model Risk Management for Agents

Read the carve-out literally, because auditors will.

SR 11-7 governed model risk in US banking for fifteen years, and by 2024 most large firms had bent it to cover machine learning. SR 26-2 supersedes it, keeps traditional ML firmly in scope, and declines to extend that scope to generative and agentic systems — the agencies' position being that these need their own framework rather than a stretched one. The UK PRA's SS1/23, in force since 17 May 2024, is drafted around the same object: a model with inputs, outputs and a definable methodology.

• Out of scope is not out of the perimeter. An agent that summarises a borrower's file for an underwriter has not escaped ECOA; an agent that triages sanctions hits has not escaped the BSA. The decision is regulated by its own statute, and the supervisory question becomes "under which of your frameworks does this thing sit?" — a question with a wrong answer of "none".
• The gap is jurisdictional, not substantive. Nothing in the carve-out says agentic systems are low risk. It says the existing MRM machinery — independent validation, effective challenge, the model inventory — was not written for them, and the agencies would rather write something new than have firms pretend the fit is good.
• "We are waiting for the guidance" is the failure mode. It is also the most common posture, because it is free until an examiner asks what compensating controls you ran in the meantime. Nothing about the promised follow-on guidance prevents a firm from applying its existing operational-risk and third-party-risk frameworks now, and every firm already has both.
• Elsewhere the answer is already written. The EU AI Act does not carve agents out of high-risk classification, so a multinational running one agent estate will be inside a binding regime in Frankfurt and inside a gap in Charlotte. Design to the stricter one — see the EU AI Act for agents.

The risk object is the action set, not the model.

Model risk management validates a function: given these inputs, the methodology produces this output, and the risk is that the output is wrong or misused. An agent is not a function. It is a policy that selects actions in an environment, and the same weights with a different tool list are a different system with a different risk profile entirely.

• Two agents on the same model are not the same model. One with read-only retrieval and one with a payment tool share every parameter and share almost no risk. Any inventory keyed on the model ID will record them as one entry and be wrong about both.
• The dangerous failure is a correct output used to take a wrong action. Classical model risk asks whether the number is right. Agentic risk asks what happened next — a well-calibrated relevance score fed into a tool that closes accounts is a control failure that no output-accuracy test detects.
• Risk scales with reachability, not with model size. Rank your agents by what they can reach and how reversible it is: read-only, draft-for-human, write-with-approval, write-unattended. That ordering predicts incident severity far better than parameter count or benchmark scores, and it is the ordering to put in the tiering policy.
• The environment is part of the system under validation. Tool descriptions supplied by third parties are prompt text you do not own and can change without a release — see third-party tool drift. A validation that inspected only your prompt validated half the system.

Re-point effective challenge at trajectories.

The three components of independent validation — conceptual soundness, ongoing monitoring, outcomes analysis — survive the transition intact. What changes is the evidence each one accepts.

• Conceptual soundness becomes a design review of the loop. Not "is this architecture appropriate for the data" but: what can this agent do, what stops it, what happens when a tool returns something adversarial, where does a human sit, and what is the blast radius of the worst single action. A vendor's model card answers none of these because none of them are properties of the model.
• Ongoing monitoring is trajectory monitoring. Track step counts, tool-call distributions, escalation rates, override rates and abstention rates — and alert on their drift, not just on output distribution. A tool-mix shift is the earliest available signal that behaviour changed, and it arrives weeks before an outcome does. See outcome vs trajectory eval.
• Outcomes analysis needs a sampled human ground truth, permanently. There is no backtest for an agent, because the actions changed the world the next observation comes from. The substitute is a standing sample of runs adjudicated by qualified reviewers on a schedule, budgeted as a permanent operating cost rather than a launch cost.
• Independence has to be constructed here. Validation cannot be the team that wrote the prompt. In practice the second line needs its own eval set, its own harness and the ability to run the agent against held-out cases without asking the builders for access — otherwise "effective challenge" means reading a deck.
• Non-determinism is a validation finding, not an excuse. Fix seeds and versions where you can, and where you cannot, report the variance: run the same case n times and record the spread of actions taken. A control that passes once and fails one time in twenty is a control you have not tested.

Fix the inventory unit before anything else.

Every MRM regime is anchored on a complete inventory, and this is where the agentic gap bites first: firms have a model inventory and no place to put an agent. Both extremes fail — recording only the base model collapses distinct systems into one row, and recording every prompt revision produces an inventory nobody maintains.

• Register the configuration, not the checkpoint. The unit is the tuple of model version, system prompt version, tool and permission set, and policy configuration. That tuple is what determines behaviour, and it is what a change-management trigger should key on — the same versioning object as rollout and versioning.
• Generate it from the runtime. A spreadsheet maintained by hand is stale within a sprint. Emit the registry from the deployment configuration so that shipping an agent without registering it is not possible — the mechanics are in agent inventory and registry.
• Name a business owner and a validator per entry. Not a team, a person. The accountability mapping matters more here than in classical MRM because the agent acts between reviews rather than only at scoring time — see accountability and roles.
• Record the tools as dependencies with owners. A third-party tool inside an agent's action set is a vendor dependency inside a decision process, and it belongs in third-party and vendor risk as well as in the model inventory.

File the agent under operational risk and stop waiting.

If the MRM framework does not claim it, something else must. In practice the workable home is the existing three-lines-of-defence structure with the agent treated as a process-with-automation, borrowing the model-risk vocabulary without pretending the agent is a model.

• First line owns the controls. Approval gates on irreversible actions, spend and step ceilings, scoped credentials, and the runbook for turning it off. These are operational controls and are testable as such — human-in-the-loop and kill switches are the two that examiners ask about first.
• Second line runs the challenge and the tiering. Independent validation as above, plus the risk-tier determination that decides how much of it applies. Tier by reachability and reversibility, per Step 2.
• Third line audits the process, including the determination memo. Internal audit's question is whether the framework assignment was made deliberately and reviewed, not whether the agent is accurate.
• Compliance owns the statutory overlay. Adverse-action reasons, disclosure, record retention and consumer-facing language are unchanged by the technology; if the agent's output feeds an adverse action, the specific-reasons requirement applies to whatever produced it.
• Do not let a vendor's assurances substitute for validation. A SOC 2 report and a benchmark card describe a supplier's controls and a supplier's evaluation. Neither is evidence about your action set on your data, which is the thing under review.

What passes an exam, and what to build this quarter.

The gap will close: the agencies said follow-on guidance is coming, and firms that already have the artefacts will be mapping them rather than building them. Four deliverables carry almost all the weight.

• A complete agent inventory generated from the runtime, keyed on the configuration tuple, with owner, validator, risk tier and action set per entry.
• A written framework determination per agent, signed and dated, saying which regime owns it and why, with a review trigger.
• A replayable record per run covering tool calls, arguments, approvals and the configuration version that produced them — retained for the longer of your model-documentation schedule and the statutory retention on the underlying decision, per retention and legal hold.
• A standing validation sample with an adjudication procedure, a target sample size derived from the population rather than from convenience, and a documented escalation path when it fails.

Related: NIST AI RMF for agents for a voluntary framework that does not carve agents out, audit trails for the record layer, governance in practice for the operating rhythm, and finance agents for the build side of the same problem.