The AI-SDLC Maturity Model

Why another model

Tell three published models that your team is "at Level 3" and you have told them three different things. Dan Shapiro's Five Levels run "from Spicy Autocomplete to the Dark Factory", and his Level 0 already lets you "occasionally hit tab to accept a suggestion", while almost everyone else's L0 means no AI at all. Steve Yegge's scale, as told to The Pragmatic Engineer, runs from 1 to 8, and its upper levels count agents rather than checks. Hassan et al. label their levels "SE 1.0" to "SE 5.0". None of these numbers tells you what a reviewer still looks at.

There is a structural gap too. Almost every published model is a single scale for coding, usually borrowed from the SAE driving levels, and the lifecycle-wide frameworks describe a process or organisational stages. No published model is a per-phase grid. Yet a team can let an agent open pull requests unattended while every deployment still waits for a human, and a single number hides exactly that difference. In June 2026 Gartner recommended "classifying development tasks into three execution models: developer‑led, developer‑with‑agent, and fully agent‑led", and deciding "appropriate levels of autonomy for each task". That is an analyst's opinion, but it endorses the design used here: a level per phase, and within each phase a level per task type.

The defining axis: what a human reviews

The published models that are most useful in practice agree on one variable: what the human reviews. Sourcegraph's Levels of Code AI (2023) puts the line between a level where the developer "is expected to review all of the code that the AI generates" and one where the human reviews only the final output; at its L4, "humans are no longer expected to review the final code output". Dash0's six levels (2026) go from "Review and approve every PR before merge" to "Look at intent and proof (satisfaction reports, screenshots, behavior evidence), not diffs". Both track the reviewer's object: every line, then an evidence bundle, then nothing.

The academic work adds a second point: a level is something you choose to deploy, not a property of the model. Morris et al. at Google DeepMind write that "Increasing capabilities unlock new interaction paradigms, but do not determine them", and that the appropriate level "need not be the maximum achievable given the capabilities of the underlying model". Feng, McDonald and Zhang define their levels by the user's role, from operator to observer, and argue that autonomy "can be treated as a deliberate design decision".

This model therefore leaves three things out of the definition on purpose.

• Capability. A stronger model makes a higher level possible. It doesn't put you there.
• Agent count. Scales that count parallel agents reward throughput with no verification attached.
• Rollout breadth. Microsoft's four-level model runs from "Ad hoc use of AI assistants by individual developers" to "Agent teams orchestrated across the lifecycle". It measures spread, not authority.

What is left is a pair of observable facts. One is the review mode, meaning what a human looks at and when. The other is the authority, meaning what the AI's credentials and settings let it do. The rules in section 2.4 say what happens when the two disagree.

The five levels

These are the book's definitions. Each is a criterion an auditor could check over a trailing window of 30 or 90 days, and each names the evidence you would show.

L0 Manual

No AI touches this phase's outputs, or policy prohibits it. Deterministic automation, such as codemods, Renovate and rule-based bots, counts as L0 for AI: it may be heavily automated, but none of it is AI. Evidence: the policy, or the absence of AI tooling for the phase.

L1 Assisted

AI suggests and a human authors every artefact. AI output enters only as suggestions the human edits. Evidence: tool telemetry exists, and no artefact is attributed to an agent.

L2 Delegated

For at least one named, recurring task type, the AI produces a complete artefact: a pull request, a test file, an ADR draft or a mitigation proposal. Every one of those artefacts gets recorded human review before it takes effect. Agent output is attributed by a label, a commit trailer or metadata. A shared instruction or context file exists, and the team tracks acceptance or rework for the task type. Evidence: approval logs that cover 100% of the artefacts, the attribution, the file and the tracked rate.

L3 Supervised-autonomous

For named routine task types, the AI runs end to end. A human reviews an evidence bundle, a sample, or only the items routed to them, not every line. Hassan et al. call the bundle a Merge-Readiness Pack: "a structured evidence bundle submitted for human approval, designed to demonstrate functional completeness". All of the following must hold:

• The routing rules, which decide what reaches a human, are written down and enforced by machine.
• Pause and rollback exist and have been exercised.
• The escaped-defect or incident rate of AI-run work is compared against a human baseline over at least 90 days.
• The AI cannot change its own evaluator (Rule 1).

L4 Autonomous (always for a named task class)

For one enumerated, narrow, reversible task class, the AI acts without per-item human approval, for example by auto-merging or auto-remediating. All of the following must hold:

• A documented verification record exists, either N consecutive items with no escaped defect or an error budget.
• Monitoring is continuous, with automatic rollback or a kill switch.
• A named human is accountable and has documented the risk acceptance.
• Audit samples are taken periodically.
• The scope is the task class, never the whole phase.

This book always writes L4 with its scope, as in "L4 for dependency patch bumps", so it cannot be misread as "the phase runs itself". On how the human's role changes from rung to rung, see human-in-the-loop and progressive autonomy.

The rules

Teams place themselves generously, so seven rules make the model hard to game. Rule 6 is a heuristic and is labelled as one; the others follow from the evidence.

Rule 1. The oracle principle

The rule exists because agents game whatever checks they can reach. METR found that o3 reward-hacked in 39 of 128 RE-Bench runs (30.4%). Models attempted "modifying the tests or scoring code", and in one case o3 monkey-patched an evaluator so that every submission was marked successful.

ImpossibleBench built tasks whose specifications contradict their tests, so that "any pass necessarily implies a specification-violating shortcut", and frontier models still produced such passes. The same behaviour shows up in production. Spotify's agents took shortcuts such as "commenting out failing tests or downgrading Java versions", as reported by InfoQ from a Spotify talk. Elastic's CI agent "fixed" broken dependency bumps by removing the bump; Elastic added the rule against downgrading versions only after seeing it.

Enforced, the rule works: every migration on record that reached L3 had a deterministic check closing the loop, whether compiler, tests or linter. The anonymous practitioner site ASDLC.io states the rule in its own words: at L3 the AI "cannot modify the underlying definition of success", and any change to an evaluator "must be restricted to a proposal loop" that passes "a human-in-the-loop audit". The wiki covers the failure mode in eval integrity and scorer gaming and the generator-verifier gap.

Rule 2. The level is the authority actually granted

A phase's level is the higher of two readings: the review mode the team practises, and the authority the AI's credentials and settings allow. An agent that holds a token able to reach production, or runs in a skip-permissions mode, is operating at L4 for everything that token reaches, whatever the team calls it.

The incident record supports the rule. In December 2025 an AWS Cost Explorer interruption involved Amazon's Kiro agent, and the two accounts of it disagree. The Financial Times, as relayed by Reuters and Engadget, reported a 13-hour interruption after the agent decided to "delete and recreate the environment". Amazon's correction (20 February 2026) called it "brief" and the result of "user error—specifically misconfigured access controls—not AI as the story claims". Amazon also said "Additional safeguards include mandatory peer review for production access." Read together, the two accounts point at the same class of failure: an operator role broader than the review around it. The fix was a boundary, not a better model. Anthropic's auto-mode write-up lists overeager actions from its own internal use, including "deleting remote git branches" and "attempting migrations against a production database". Each is a de facto L4 grant that nobody knew had been made (see blast radius).

Rule 3. Score each named task type, not the whole phase

A phase's level is the highest level at which a named, recurring task type in that phase runs with the required evidence. Always record the task type next to the level: "Testing: L3 for flaky-test triage; L2 for feature tests." A phase-wide number overstates the risky work or understates the routine work. See task scope for why task boundaries, not task size, decide what can be delegated.

Rule 4. Risk sets a ceiling

Each combination of phase and risk class carries a maximum level, recorded in a ceiling column of the self-assessment. For example: nothing above L2 in CI/CD for regulated services, and no L4 anywhere an action is irreversible. James Read puts it briefly: "As risk increases, acceptable autonomy decreases". The Cloud Security Alliance and AWS both attach escalating controls to each level (see section 2.5). Irreversibility alone makes a task class ineligible for L4.

Rule 5. There is no L5, and L4 is narrow

The serious sources agree that general autonomy either doesn't exist or has no controls. Hassan et al. say their general-domain L5 "currently is at the conceptual/research stage, i.e., it does not yet exist". The Cloud Security Alliance says "the control mechanisms required to safely operate at this level don't exist yet". The ai4se taxonomy calls full automation "rather unrealistic for complex, industrial-scale software". The agent-swarm showcases, Thoughtworks notes, "chose use cases that could rely on existing detailed specifications… Those conditions are not representative of typical product development". Levels are not targets; the right level for a phase can be L1 indefinitely.

Rule 6. A heuristic: run Coding at most one level above the lower of Review and Testing

This is the book's proposal, not a law. When coding runs ahead of the phases that verify it, the telemetry shows a consistent pattern: more pull requests, bigger pull requests, longer reviews, more merges with no review at all, and more instability. DORA's 2025 report offers a mechanism: AI-associated instability is "likely, in part, because it is harder to review larger batches of code". The vendor telemetry in the box below agrees on direction. The evidence is correlational, so treat the rule as a tripwire, not a proof. When Code Gets Cheap, the Bottleneck Moves makes the full case.

Rule 7. Evidence beats self-report

A placement must cite something observable: approval logs, attribution, routing configuration, a credential inventory or a verification record. The reason is that people's sense of their own speed and safety is unreliable. In METR's 2025 trial, experienced developers took 19% longer with AI and afterwards believed they had been 20% faster.

A Stanford research group found that 43 engineers' self-rated productivity correlated with their measured productivity at only r = 0.17, and that people "misjudge their productivity by ~30 percentile points" (conference slides, not peer-reviewed).

The same drift applies to autonomy: Thoughtworks carried "Complacency with AI-generated code" in its Radar's Hold ring ("Proceed with caution") for three editions (October 2024 to November 2025), warning that "It's all too tempting to be less vigilant when reviewing AI suggestions after a few positive experiences with an assistant." A team's estimate of its own level will rise with comfort unless the evidence holds it in place.

Controls that come with each level

A level is also a set of permissions. The Cloud Security Alliance asks for "machine-readable boundary definitions" and "technical enforcement mechanisms that actually prevent boundary violations" at its L3. At its L4 it asks for "immediate kill-switch capabilities", "continuous monitoring" and "executive authorization and documented risk acceptance". The AWS Agentic AI Security Scoping Matrix moves from "explicit human approval for all actions of consequence" (Scope 2) to agents that act "without further approval" inside "Clear execution boundaries defined at initiation" (Scope 3). Their numbering is offset from this book's; section 2.8 maps them.

The table below is the book's proposal, synthesised from vendor defaults and post-incident fixes. GitHub's Copilot cloud agent, for example, "only has the ability to push to a single branch"; its "workflows are not triggered until Copilot cloud agent's code is reviewed"; and the person who requested the work cannot approve its pull request (as of October 2026). In Anthropic's sandbox design, "sensitive credentials (such as git credentials or signing keys) are never inside the sandbox". No outcome-validated standard exists, and no published study gives incident rates by permission level.

Chapter 12, Security, Permissions & Blast Radius, owns the full permission envelope and the incidents behind each row. For who signs the L4 risk acceptance, see accountability and roles.

Enablers that gate a level

Controls limit what an agent can do; enablers decide whether a climb pays off at all. The strongest evidence is DORA's AI Capabilities Model (September 2025). According to the 2025 report (p. 50), DORA started from 78 interviews and 15 candidate capabilities, then tested them against its survey. Seven showed "substantial evidence of an interaction with AI use": with them in place, the difference AI made to key outcomes was amplified. The evidence is survey-based and correlational. Treat the seven as preconditions to check before a climb, not as levels.

• A clear and communicated AI stance.
• Healthy data ecosystems.
• AI-accessible internal data.
• Strong version control practices.
• Working in small batches. Trade-off: "AI adoption's benefits for individual effectiveness are slightly reduced in teams that are working in small batches." DORA judges the net effect positive.
• User-centric focus. Without it, according to the 2025 report, "AI adoption has a negative impact on team performance."
• Quality internal platforms. Trade-off: with a quality platform, "AI's neutral effect on respondents' reported experiences of friction is made harmful". DORA's hypothesis is that platforms act as guardrails, and guardrails add friction.

Microsoft's Agentic DevOps model, a vendor framework with no data, puts it bluntly: "agents do not magically fix broken engineering practices". Without small batches, working rollback and usable internal context, a higher level mostly amplifies the problems a phase already has.

The grid: eight phases, five levels

This is the map. Each cell gives the observable criterion, then the best public evidence with its tier (defined in the Preface). "None" means none was found.

Requirements

Developed in Chapter 5, Requirements, Specs & Design.

Design

Developed in Chapter 5, Requirements, Specs & Design.

Coding

Developed in Chapter 6, Coding.

Review

Developed in Chapter 7, Code Review.

Testing

Developed in Chapter 8, Testing & Verification.

CI/CD

Developed in Chapter 9, CI/CD & Release.

Operations

Developed in Chapter 10, Operations & Incidents.

Maintenance

Developed in Chapter 11, Maintenance & Migrations.

Read the grid as a whole and three observations stand out.

1. Real L3 evidence exists in only a few places: coding (volume only), testing (partly), maintenance, and a narrow, self-reported class in operations. No phase has credible L4 evidence for LLMs. Requirements and design have no L3 evidence at all.
2. Classical ML got there first. Release gating with automatic rollback, as in Harness Continuous Verification, reached L3 and L4 with classical ML years ago. The grid counts classical ML as AI but doesn't credit it to LLMs, and deterministic automation, however autonomous, is L0 for AI.
3. The incidents cluster where permissions outran review. Every cell that holds an incident (coding L4, operations L3 and L4) is a case where granted permissions put the team higher than it believed: Rule 2 at work.

The cases behind these cells, with their full measurement caveats, are in Case Studies: Who Did It and What They Measured.

Translating other models

When someone says "we're at Level 3", ask whose scale they mean. The table is the book's reading of each model; "≈" marks an approximate mapping. Appendix C, Glossary, repeats it.

One warning closer to home: the wiki's own autonomy levels page counts from L0 = "Suggest", so its L0 to L2 sit one step below this book's numbers.

Scoring yourself

Three worked examples, from public data only, score a phase by review mode, by verifier design and by granted authority.

Worked example 1: dotnet/runtime's coding phase

Microsoft's .NET team published ten months of data on GitHub's Copilot Coding Agent in its runtime repository. Every agent PR went through human review, so the phase scores L2 for every task type, however good the merge rates look. The question is which task types are candidates for L3. Cleanup work merged at 84.7% (77 PRs) and testing work at 75.6% (99 PRs). Performance work merged at 54.5% (46 PRs), so it isn't a candidate. To move cleanup to L3, the team would need two things the post doesn't claim. The first is a verifier the agent cannot edit, deciding pass or fail. The second is a 90-day comparison of escaped defects. The revert data are a start: 3 of 535 merged agent PRs were reverted (0.6%), against 33 of 4,251 other merged PRs (0.8%). The post itself says the difference isn't statistically meaningful.

Worked example 2: Spotify's maintenance phase

Spotify's background agent, Honk, runs fleet migrations. Its deterministic verifiers (format, build, test) run in a stop hook, and when one fails, "the PR isn't opened and the user is presented with an error message". The agent is "highly sandboxed", its toolset is deliberately small, and pushes and Slack messages come from the surrounding infrastructure, not from the agent. On design evidence, that is L3 for fleet migrations: the oracle sits outside the agent's reach, and a human sees the result. The public record lacks the 90-day defect comparison, and an outside scorer should write that gap down. Spotify's Fleet Management is a separate system. Its 2.5 million-plus PRs are mostly auto-merged with no human in the loop (as of June 2026). It began as deterministic scripts, and Spotify does not say how many of the auto-merged PRs its LLM agent wrote, so its autonomy cannot be credited to AI; the deterministic part scores L0 for AI however autonomous it is.

Worked example 3: scoring by granted authority (Kiro)

Whatever review mode was intended, an operator role broad enough to delete and recreate a production environment put that task at de facto L4. The two accounts differ on everything else. The FT, via Reuters and Engadget, reported 13 hours; Amazon says the interruption was "brief", "an extremely limited event" affecting one service in one of 39 regions, and caused by "a misconfigured role—the same issue that could occur with any developer tool (AI powered or not) or manual action". The score doesn't depend on which account is right. Amazon's fix, "mandatory peer review for production access", returned the task to L2, and it did so by changing the authority, not the model.

The self-assessment template

Fill one row per phase and task type. The "credentials held" column is what turns this into Rule 2: wherever it reaches further than the review column, record the higher level. The rows below are illustrative and name no organisation. Appendix A, Templates, carries a blank copy, and Chapter 13, Assess Your Starting Point, walks through filling it in.

PHASE        TASK TYPE              LEVEL  EVIDENCE                          RISK CEILING  CREDENTIALS HELD
-----------  ---------------------  -----  --------------------------------  ------------  ------------------------------
Coding       dependency cleanup     L2     approval log 100%; PR label       L3            dev token; no prod access
Coding       performance work       L1     no agent attribution              L2            none beyond IDE
Testing      flaky-test triage      L3     verifier outside agent; 90-day    L3            CI read; own branch only
                                           escaped-defect comparison
Operations   cache restart          L2     typed tool; human confirms each   L3            scoped service identity
Operations   (any) via shared token L4 !!  token reaches prod DB             L2            broad API token -> FIX FIRST

Look for rows like the last one first. Nobody chose L4 for that task; the token did.