When Engagements Fail

What this phase is for, and how it fails

This is the one chapter in Part II that is not a phase on the calendar. A failing engagement can show up in any week, from the pilot terms to months after go-live. Each earlier chapter makes one kind of failure less likely; this one is for when it happens anyway. The reasons repeat:

• The wrong problem. The request was a solution, and the problem behind it was never found (Discovery).
• No data. The data the work needs does not exist, cannot be reached, or cannot be reached in time.
• No owner. Nobody on the customer side owns the result once it works, so it works and nobody uses it.
• A moving sponsor. The person who wanted the outcome changes jobs, or changes priorities.
• An unagreed threshold. Nobody fixed what "good enough" means, or it was fixed and the work cannot reach it (Scoping and the First Win).

None of these is a model failure. TNW reports the estimate of Colin Jarvis, who heads forward deployed engineering at OpenAI, that in about 80% of cases where a company struggles with enterprise AI, the problem is deployment; in the other 20%, customers say the model cannot yet do the job (TNW, September 2026; T3, a journalist relaying a vendor's estimate, with no method behind the number). Read it as an informed opinion.

How failure is handled goes wrong in three ways. The slow fade: meetings thin out, nobody says the word, and the engagement drifts until a renewal date forces the question. The late kill: someone finally calls it, after the money and the goodwill are gone. The blame post-mortem: the review is written to decide whose fault it was, so the next attempt starts without the lesson and without the sponsor.

The slow fade is the most common, and the best-documented example is public. New York City's MyCity business chatbot was released in the autumn of 2023. Testing by The Markup and THE CITY in 2024 found it would "confidently provide incorrect and potentially harmful information", including suggesting landlords could discriminate against tenants with Section 8 vouchers. The city said it would improve. A new mayor called it "functionally unusable", and The Markup's update of 4 February 2026 reported that the "beta test has ended" (The Markup, 2026). The testing is T1, the journalists' own measurement; the cost, "around half a million dollars" in the mayor's words, is T3. The lesson is the timeline: the evidence was public for nearly two years before anyone with authority called it.

One honest gap. There is no public, first-hand account of an FDE deliberately ending a customer engagement and describing the criteria used; the research behind this book looked and found none. Failed engagements end quietly, under NDAs. The playbook below is built from adjacent evidence: practitioner interviews, public shutdowns and reversals, and one study of successes that asked about the failures before them. Where it turns from evidence to recommendation, it says so.

The composite: the expansion that stalled

Three things in that beat are the chapter. Nobody cancelled anything; the failure showed only as missed meetings, an unnamed owner and a stalled labelling job. The rule was written before anyone needed it, so it judged no one's commitment. And walking away from property was not walking away from the customer.

Don't argue from base rates

Sooner or later someone reaches for a failure rate: "95% of these fail anyway", from your own account team or from the sponsor's boss pulling the budget. The rates in circulation are weaker than they look, and an FDE who repeats them will be caught out.

RAND, 2024 (T1, with a caveat). RAND interviewed 65 experienced data scientists and engineers about why AI projects fail (RAND RR-A2680-1). It measures what practitioners believe, not outcomes. Two findings matter: "Eighty-four percent of our interviewees cited one or more" leadership-driven root causes "as the primary reason that AI projects would fail", and 14 of the 50 industry interviewees said senior leaders underestimated how long an effective model takes. Its "more than 80 percent" failure line is not its measurement but a "By some estimates" citing Fortune: an echo, as Discovery noted.

MIT NANDA, 2025 (T2). "95% of organizations are getting zero return" comes from a file named v0.1 and headed "Preliminary Findings" (MIT NANDA, July 2025). It rests on interviews with 52 organisations, a survey of 153 senior leaders and a review of over 300 public initiatives. It defines success as "deployment beyond pilot phase with measurable KPIs", measures return six months after the pilot, and calls its figures "directionally accurate based on individual interviews rather than official company reporting". R. Scott Raynovich at Futuriom wrote that the 95% "is presented in one sentence" with no detail on its source, and quoted Wharton's Kevin Werbach: "there appears to be no further support for the 95% claim" (Futuriom, August 2025; T3).

Gartner (forecasts, not measurements). In July 2024 Gartner predicted that "at least 30% of generative AI (GenAI) projects will be abandoned after proof of concept by the end of 2025", citing "poor data quality, inadequate risk controls, escalating costs or unclear business value". In June 2025 it predicted that "over 40% of agentic AI projects will be canceled by the end of 2027", alongside a January 2025 poll of 3,412 webinar attendees about how much they had invested (Gartner press releases, July 2024 and June 2025; T4, analyst forecasts). Both get re-cited as outcomes, and this book's research found no follow-up measuring whether the first came true.

So when a rate comes up in the room, answer with the engagement, not a better statistic: "Nobody has a reliable failure rate for projects like this one. What we do have is our own five conditions, and here is where each stands." A base rate cannot tell you whether this engagement is failing. Only its own signals can.

Five failures, and the question that tests each one

Each failure signals weeks ahead, and each has a question you can ask this week that turns a feeling into a yes or a no.

The wrong problem. The signal is a good demo after which nobody asks for the next step. It is RAND's first root cause, covered in Discovery (T1). The test: "If this reaches the threshold, which number in your next operating review moves?" If the answer names a different number from the one in the scope, or none, the scope is serving a problem the business has stopped caring about. Go back to discovery, not to the model.

No data. The signal is a date. If production-like data access has not arrived by the end of week one, the pilot is running on fixtures and hope. RAND's second root cause is organisations that lack "the necessary data to adequately train an effective AI model" (T1). For an FDE the commoner form is data that exists but cannot be reached in time. The test: "Who must approve this access, and has each of them seen the request?" The pilot terms in Before the Engagement stop the clock when access slips; use that clause, and give the sponsor the names.

No owner. The signal is a question nobody answers: who runs this when we leave? Stanford's study of successful deployments includes a semiconductor company whose earlier agents "worked in demos but not in production". The diagnosis: "The problem was not technical"; there were "no shared standards, no accountability for adoption" (Pereira, Graylin and Brynjolfsson, 2026; T2, self-reported in interviews, in a report marked draft). The rescue there was a leader who escalated to the CEO. The test: "When this breaks after we leave, whose phone rings, and do they know it yet?" A team name is not an answer; a person is. Without one, a working system is switched off at the first budget review.

A moving sponsor. The signal is attendance. RAND's interviewees described organisations where "senior leaders rapidly switch their priorities every few weeks or months", so that projects in progress "can be discarded before they have the opportunity to demonstrate real results". One interviewee put the cost precisely: "Often, models are delivered as 50 percent of what they could have been" (RAND, T1). A sponsor who has moved rarely says so. They delegate the steering meeting, stop reading updates, then ask whether the work could be "paused for a quarter". The test, asked by the lead and not by email: "Is this KPI still the one you are judged on this half? If not, which one is?" Most sponsors answer that honestly, because it is about them.

An unagreed or unreachable threshold. The signal is a number that never settles, or one that settles below the target and stays there. The Evals chapter tells the McDonald's drive-through test: accuracy reported "in the low 80% range" against a "95%-plus" target, and an end to the test roughly two and a half years after the partnership began (Nation's Restaurant News, 2024; T3, the accuracy figure second-hand). Evals sets a date to re-measure; this chapter adds the harder one. The test: "If we are still below the threshold on that date, what do we do?" If nobody will answer, the climb has no end, and that is the failure.

One trap sits after delivery rather than before it: success declared on an automation metric, then reversed. Klarna announced in February 2024 that its assistant handled "two-thirds of Klarna's customer service chats" in its first month, "the equivalent work of 700 full-time agents" (Klarna; T2, self-reported). About fifteen months later its CEO told Bloomberg the result had been "lower quality", and that "investing in the quality of human support is the way of the future for us" (Entrepreneur, May 2025; T3, relaying one interview). Note the echo: that report gives the original share as 75%, not two-thirds. Commonwealth Bank of Australia announced 45 customer-service roles would be replaced by an AI "voice bot", then reversed the cuts, saying its assessment "did not adequately consider all relevant business considerations" and "this error meant the roles were not redundant". The union said call volumes had been rising (Information Age, August 2025; T3). Neither was an FDE engagement. Both show the risk to name in writing: the agent's share of the work is not evidence the work is done well.

The wiki's failure taxonomy and triage page does the same job for an agent's runtime failures. This list is the engagement-level version, and Appendix C cross-references the two.

Call it while you can still act

A post-mortem is usually run on a corpse. Run this one on a patient, while there is still time to rescue it, and write the conclusion down before anyone has a reason to soften it.

When. On the second signal, not the fifth. One missed steering meeting is noise. A missed meeting plus an unnamed threshold owner is a pattern. Two signals from the grid at the end of this chapter inside any four-week window are enough to schedule the review. This is this book's rule of thumb, not a measured trigger.

Who calls it, and to whom. Usually the FDE, who sees the signals first: the labeller who stops replying, the data request stuck in a queue, the demo nobody senior attends. Call it to your own lead first, in writing, the same day, and let the lead take it to the sponsor; an FDE who goes straight to the sponsor turns a vendor decision into a personal complaint. Four lines are enough: the scope, two dated signals, the conditions you think are missing, and the ask (a post-mortem before the next invoice or milestone). The skill map in Chapter 3 puts calling a failure early at the top level of judgment, because few people do it.

Against what. The signed documents: pilot terms, scope, threshold, exit criteria, and the rule. Not impressions of commitment. If the terms were never written, that is the first finding.

Without blame. Here the evidence is unusually direct. Stanford's study of 51 successful deployments found that "61% of successful projects included at least one prior failure", that "in every case where we could identify whether the same executive sponsored both the failed and successful attempts, the answer was yes", and that "in none of the cases we examined was anyone punished for a failed AI initiative" (Stanford, T2). The sample is successes only, and the authors name the limitation: "selection bias toward positive outcomes". So it cannot show that blameless reviews cause rescues. It does show that the projects which came back came back with the same sponsor, who had not been made to pay for the first attempt. Embarrass the sponsor and you spend the asset most likely to bring the work back.

Rescue or walk: the decision rule

Every engagement rests on five load-bearing conditions, one for each failure above:

1. a problem the sponsor's business still cares about;
2. a working path to the data;
3. an owner for the result on the customer side;
4. a sponsor who owns the number and still has time for it;
5. a threshold that is agreed, owned and reachable.

Write it down at the start, in the scope or the pilot terms. Continue if all five hold. Reset if one or more are missing and, for each missing one, a named person on the customer side commits to restoring it by a date. Walk if any missing condition has nobody willing to put their name and a date next to it. This is this book's recommendation, not a tested method. The evidence behind it is RAND's account of how leadership-driven failures play out and Stanford's account of how rescued projects differed.

Two details decide whether the rule bites. The restorer must have authority over the missing thing: a sponsor's deputy promising the head of property's time is not a restorer. And the date must be close, within a few weeks and before the next invoice or milestone, so that a missed date triggers the rule rather than another meeting about it.

The rule also changes the conversation. "We think you've lost interest" is an accusation. "Can a threshold owner be named by the end of the month?" is a question with a yes or a no, against criteria the sponsor already agreed. If the sponsor says no, they have made the decision; the lead's job is to make that easy.

What a reset looks like. Stanford's clearest rescue is a recruiting system at a professional-services firm. The first attempt "assumed AI would fix broken processes". The second changed three things: "the CEO took ownership rather than delegating to the CTO", "they fixed the process before applying AI", and "they targeted genuine pain". In an executive's words: "It wasn't 'Hey, this would be great.' It was 'I'm drowning.'" (Stanford, T2; the outcome figures in the case are self-reported). Each change restores one of the five conditions: an owner with authority, the right problem, and a sponsor who needs the result. A reset that changes only the model or the prompt has restored nothing.

What a reset is not. "Keep going and see." A reset without a re-check date is a slow fade with paperwork.

When walking is right. Walking is right when the missing conditions cannot be restored by anyone with the authority to restore them. It saves the customer paying for work that cannot land. Scoping quoted RAND's advice to commit a team to one problem for at least a year. Read it backwards here: if nobody at the customer will commit a named person for the next few weeks, they are not committing to the year, and the scope should stop. Walk from the scope, not the customer. Anything already in production stays, and so does the relationship.

Walking away well

How you leave decides whether you can come back.

• Leave what works running, and owned. If any part of the engagement is in production, it stays there with the customer's team as its owner, under the runbook from Production and Handoff.
• Hand over the work in progress. Labelled data, fixtures, draft scopes and the eval harness belong to the customer's next attempt, whoever runs it.
• Stop billing on a date, in writing. The lead and the account executive tell the sponsor when billing ends. A walk without that date is a pause.
• Write the trigger for coming back. Put the abandoned scope on the "not now" list with a concrete trigger: a named owner, a labelled sample, a sponsor with time.
• Keep the sponsor. Send the post-mortem to the sponsor before anyone else reads it, and make it say what worked as clearly as what did not. Stanford's evidence says the next attempt, if there is one, will probably have the same sponsor.

Three customers stalling at the same integration is a product gap, not three bad engagements. Feeding the Product covers turning field evidence into a brief, and a failed engagement is good evidence, because nobody has an incentive to dress it up.

The template: the engagement post-mortem

Section 1 comes from the signed documents, including the rule, so the rule is fixed before anyone fills in section 3. Run it when two signals appear, or after the engagement ends.

ENGAGEMENT POST-MORTEM                            version / date: ____
Run: during | after the engagement     Facilitator: ______________

1. WHAT WE AGREED (copy from the signed terms and scope)
   Sponsor and KPI:       _________________________________________
   Scope and threshold:   _________________________________________
   Threshold owner:       _________________________________________
   Exit criteria:         _________________________________________
   Decision rule:         continue if all five conditions hold;
                          reset if each missing one has a named
                          restorer + date; walk if any has none

2. WHAT HAPPENED (dated facts, no adjectives, no names to blame)
   when: ______  ___________________________________________________
   when: ______  ___________________________________________________
   when: ______  ___________________________________________________

3. THE FIVE CONDITIONS        At start    Now         Evidence
   Problem still matters      yes/no/?    yes/no/?    ______________
   Data path working          yes/no/?    yes/no/?    ______________
   Owner for the result       yes/no/?    yes/no/?    ______________
   Sponsor owns the number    yes/no/?    yes/no/?    ______________
   Threshold agreed, owned    yes/no/?    yes/no/?    ______________

4. MAIN CAUSE (one only: the one that explains the others)
   wrong problem | no data | no owner | moving sponsor |
   unagreed threshold | too hard for AI | other: _______________

5. WHAT STILL WORKS, AND WHO OWNS IT (a person, not a team)
   - system / artefact: ______________  owner: ____________________
   - system / artefact: ______________  owner: ____________________

6. DECISION:  continue | reset | walk          decided by: ________
   If reset:  condition: ________  restorer: ________  by: ________
              re-check date: ________
   If walk:   billing stops: ________
              handed over: ________________________________________
              return trigger: _____________________________________

7. WHAT GOES BACK TO PRODUCT
   - _______________________________________________________________

8. SIGN-OFF
   Sponsor:        __________   date: ______
   Vendor lead:    __________   date: ______
   FDE:            __________   date: ______

Here it is filled from the composite, for the property expansion. Every figure is a range or a relative date, as the composite rules require.

ENGAGEMENT POST-MORTEM: property expansion      version 1 / month 8
Run: during the engagement             Facilitator: vendor lead

1. WHAT WE AGREED (copy from the signed terms and scope)
   Sponsor and KPI:       COO; claims cycle time
   Scope and threshold:   property FNOL triage, propose only;
                          threshold to be set by head of property
   Threshold owner:       head of property claims (no number yet)
   Exit criteria:         no labelled set or threshold within a
                          few weeks of scope sign-off
   Decision rule:         as written in the month-6 scope

2. WHAT HAPPENED (dated facts, no adjectives, no names to blame)
   when: month 6  property scope signed; labelling plan agreed
   when: month 7  board cost programme; COO takes it on
   when: month 7  steering meeting moved twice, then skipped
   when: month 7  property adjusters moved to backlog; a
                  fraction of the sample labelled
   when: month 7  check: exit criterion met; call deferred
   when: month 8  finance asks for auto figures to justify
                  adjuster cuts

3. THE FIVE CONDITIONS        At start    Now         Evidence
   Problem still matters      yes         ?           not in the
                                                      cost plan
   Data path working          yes         yes         replica OK
   Owner for the result       yes         yes         claims
                                                      director
                                                      (all lines)
   Sponsor owns the number    yes         no          meetings
                                                      skipped
   Threshold agreed, owned    pending     no          no number
                                                      after weeks

4. MAIN CAUSE (one only: the one that explains the others)
   moving sponsor

5. WHAT STILL WORKS, AND WHO OWNS IT (a person, not a team)
   - auto FNOL triage (live)    owner: claims director (behaviour);
                                platform ops lead (service)
   - auto eval harness          owner: claims director

6. DECISION:  walk (property only)             decided by: COO + lead
   If reset:  offered: narrower property slice, owner and labelled
              set within a few weeks; COO could not name an owner
   If walk:   billing stops: end of month 9
              handed over: partial labelled set, fixtures, draft
              scope, labelling guide
              return trigger: named threshold owner and a fully
              labelled property sample

7. WHAT GOES BACK TO PRODUCT
   - labelling capacity, not data, was the bottleneck: ship a
     labelling plan with the triage template
   - auto metric is speed, not staffing: say so in every readout

8. SIGN-OFF
   Sponsor:        COO             date: month 9
   Vendor lead:    lead            date: month 9
   FDE:            FDE             date: month 9

Two things in the filled version are deliberate. The cause is "moving sponsor", although the threshold was missing too, because the threshold went missing when the sponsor's attention did: pick the cause that explains the others. And section 6 records the reset that was offered and refused, so when property comes back, the reason it stopped is already in writing.

Failure signals

Any two inside four weeks are enough to schedule the post-mortem. Most look like scheduling problems, which is why they are ignored.

Ready to move on

The engagement, or the failed part of it, is closed properly when every box is ticked:

• The decision (continue, reset or walk) is written down, with the rule it was taken under.
• A reset has a named restorer, a date and a re-check date for every missing condition.
• Anything in production has a named owner on the customer side and a runbook they have used.
• Work in progress has been handed to the customer, not held back, and billing for the abandoned scope has a stop date.
• The sponsor has read the post-mortem before anyone outside the account team.
• The abandoned scope is on the "not now" list with a concrete return trigger.
• What the failure teaches the product has been filed where the product team will read it.