Discovery
The problem a customer names on day one is a guess about their own business, and by practitioners' account wrong guesses kill more AI projects than anything else. Discovery replaces the guess with an observed problem: the lever leadership actually cares about, and the workflow where that lever lives. It happens on site, by watching the work, before anyone picks a model. Its output is one page the sponsor signs, and everything after it is built against that page.
What discovery is for, and how it fails
The request that arrives with an engagement is almost always a solution: "a chatbot", "an agent for our support queue", "something with GenAI for the underwriters". A solution is a hypothesis about a problem, and nobody has tested it yet. Discovery is the test. It ends when you can write down, in the customer's own numbers, what is slow or expensive or wrong, where it happens, who owns it, and which of the business's levers moves if it gets better.
The strongest evidence that this is where engagements die comes from outside the FDE world. In a 2024 report built on interviews with 65 experienced data scientists and engineers about why AI projects fail, RAND lists five root causes, and the first is that "industry stakeholders often misunderstand—or miscommunicate—what problem needs to be solved using AI" (RAND RR-A2680-1, T1). The report's own recommendation is blunter: misunderstandings about "the intent and purpose of the project cause more AI projects to fail than any other factor."
Grade that carefully. RAND is independent and primary, which is why this book files it as T1, but what it measured is what practitioners believe causes failure, in interviews, about AI and machine-learning projects generally rather than language-model deployments specifically. Leadership-driven failures, of which the wrong problem is the first form, and data problems were the two causes "cited spontaneously by more than one-half of the interviewees". It is the best evidence there is. It is not a measured failure rate, and anyone quoting "RAND says 80% of AI projects fail" is quoting a Fortune estimate that RAND repeats, not a RAND finding.
RAND's example of the wrong problem is worth keeping in your head, because it is so ordinary. Leaders "may say that they need an ML algorithm that tells them the price to set for a product—but what they actually need is the price that gives them the greatest profit margin instead of the price that sells the most items." Nobody lied and nobody was stupid. The team built what was asked, and such errors "become obvious only after the data science team delivers a completed AI model and attempts to integrate it into day-to-day business operations." That is the failure discovery exists to move forward by months: from integration, where it costs the engagement, to week one, where it costs a conversation.
The FDE version of the same finding is softer evidence but closer to the job. Colin Jarvis, who runs forward deployed engineering at OpenAI, told the Pragmatic Engineer that "often what the customer describes in scoping doesn't match the data/system reality on the ground" (Pragmatic Engineer, 2025-08-12, T3: a practitioner quoted in secondary reporting). Tryolabs, a consultancy that sells forward deployed work, puts it as a rule: what to build "has to be discovered, from the inside", and "Most of the time nobody can hand you this on day one" (Tryolabs, 2026-06-30, T4: a vendor's opinion).
Discovery fails in three recognisable ways. It is skipped, because the request looked clear and the contract named a deliverable. It is done at the wrong altitude, in meeting rooms with executives, so the team learns what leadership thinks the work looks like rather than what it is. Or it is done well and then not written down, so the observed problem lives in the FDE's notes and the build drifts back toward the stated one the first time a stakeholder repeats the original request. The wiki's page on failure taxonomies and triage makes the same point about agents: label the first step that went wrong, because that is the one with a fix. In an engagement, the first step that goes wrong is usually this one.
The insurer, weeks one and two
The running composite picks up where Before the Engagement left it: a 30-day paid pilot, one KPI the COO owns, and a deliverable that still reads "customer chatbot" because that is what sales wrote down.
The request. The insurer is a regional property-and-casualty carrier with a few thousand staff, an on-prem claims system, a separate document store and a strict model-risk function. The request on paper, which came through IT and sales, is a chatbot for policyholders. In the first interview the COO does not mention policyholders at all; she talks about claims cycle time, which her board has asked about. The FDE writes "chatbot" in the stated-problem box and "cycle time?" next to it with a question mark. This is RAND's first root cause in miniature: the request is a solution, and the problem behind it has not been said aloud yet.
Week one: the meeting rooms. The claims director, the head of the contact centre, an IT lead for the claims platform and a model-risk manager each describe a different problem. The contact centre wants fewer "where is my claim?" calls. The claims director wants fewer files landing with the wrong team. IT wants nothing that writes into the claims system. Model risk wants nothing that decides. Every one of them is right about their own corner, which is the pattern Sierra describes in a deployment with "over 20 internal stakeholders and no APIs".
Week one, later: the floor. The FDE asks to sit with adjusters, not to interview them. For a few days the FDE watches first-notice-of-loss intake on auto claims. Photos, police reports, repair estimates, medical notes and long policyholder emails arrive in one shared inbox and a scanning queue. Adjusters open each one, decide what it is and which claim it belongs to, and route it. Asked to estimate, adjusters put this sorting at somewhere between a third and a half of their day, and the FDE's own tally of the documents passing through the queue during the visit lands in the same range. Nobody had written that number down before. This beat dramatises Jarvis's warning that what the customer describes in scoping "doesn't match the data/system reality on the ground."
Week two: the connection. The FDE takes a sample of contact-centre call reasons back to the claims director. Most status calls are about claims that have not had a first action yet, and the commonest reason for no first action is a document still sitting unsorted or misrouted. The chatbot would answer the call; it would not shorten the wait the call is about. The lever is time-to-first-action, and it lives in document triage at intake. This mirrors the OpenAI semiconductor engagement, where a team embedded on site for a couple of weeks before choosing verification, the work that took most of engineers' time.
The sign-off. Midway through week two the FDE puts one page in front of the COO: stated problem, observed problem, the evidence, and the lever. She crosses out "chatbot", writes "not now" beside it, and signs the observed version. Property claims, which have the same problem, go on the "not now" list too. The scope chapter starts from this page. The pattern is RAND's first recommendation: technical staff must understand "the project purpose and domain context" before they build.
Notice what the FDE did not do: pick a model, demo anything, or argue about the chatbot in week one. The re-framing landed because cycle time was already the COO's KPI. Discovery rarely overrules a sponsor. It shows her that the lever she already cares about sits somewhere other than where the request pointed.
Start with the business lever, not with AI
The first conversation should not be about AI at all. According to TNW's report of a 2026 talk, every OpenAI engagement "starts with a two-day visit", and the team "asks business leaders to ignore AI and name the biggest levers in their business" (TNW, 2026-09-23, T3: secondary reporting of one practitioner's description of his own practice). Take the instruction, not the format. The principle: the sponsor names the lever before anyone names a technique.
A lever is a number the business already watches and someone senior is paid to move: cycle time, cost per case, revenue per account, loss ratio, time to hire, scrap rate. If the request cannot be connected to one of those, it is a technology looking for a use, and RAND's third root cause, a focus on "the latest and greatest technology" over real problems, is waiting for you. RAND's interviewees also described leaders who demanded machine learning for problems that "could have quickly been captured by a few simple if-then rules". Starting with the lever is how you find out early whether this is one of those.
The job posts of the companies that hire FDEs ask for the same habit. Palantir tells applicants that its projects "often start with an open ended question like 'Why are we delaying so many flights?' or 'How can we better identify instances of money laundering?'" (Palantir FDSE posting). Glean's founding FDE post says: "Before you write a line of code, you go deep on the business. What's structurally broken? Where is the biggest concentration of unsolved value?" (Glean posting). Both are T4 as evidence of how engagements actually go. They are primary evidence of what employers ask for, and what they ask for is a question about the business, not about a model.
In practice, open with three questions to the sponsor and write the answers in her words:
- Which number would you most like to move this year? If she names several, ask which one she reports upward.
- Where do you think that number is lost? This is her hypothesis. Record it; do not argue with it yet.
- What would you accept as proof that it moved? This is the seed of the success metric that Scoping and the First Win will fix, and the baseline you will have to measure.
Then translate the original request into the lever. "A chatbot" becomes "fewer status calls", which becomes "claims wait too long for a first action". Each translation is a hypothesis you now go and check.
Go where the work happens
The lever tells you what matters. Only the floor tells you where it is lost. The best public FDE example of the difference is OpenAI's engagement with an unnamed European semiconductor company, described by Jarvis on Altimeter's podcast (Altimeter, 2025-11-26, T2: the vendor describing its own engagement, customer unnamed). The customer's request was deliberately open: look across the whole value chain and "take the biggest areas of… waste". The team "embedded on site, spent like a couple of weeks just like understanding their business in a lot of detail", across design, verification and post-ship performance measurement. They chose verification, where engineers "have to spend like 70 80% of their time like you know just like fixing bugs" and maintaining compatibility with old chips. The detail that makes it concrete: tests run overnight, and each engineer arrives in the morning to a few hundred bugs.
Be precise about what this case proves. The customer did not hand OpenAI a wrong problem; it handed over an open one, which is rarer and easier. What the case shows is the method: weeks on site, the whole chain mapped, and the target chosen by where engineers' time actually went rather than by what was most demonstrable. The reported outcome, 20 to 30% efficiency savings in the first divisions against a 50% target, is self-reported with no method given, so it is weak evidence that the choice paid off. A secondary summary of the same talk says the embed lasted "several weeks" (ZenML, T3); the recording says "a couple of weeks". Use the primary.
Why the floor and not the meeting room? Nabeel Qureshi, a former Palantir FDE who was at the company from 2015 to 2023, says going on site was meant to capture "the tacit knowledge of how they work", as opposed to "the flattened 'list of requirements' model that enterprise software typically relies on" (Qureshi, 2024-10-15, T4: one former employee's account). Tacit knowledge is the part nobody puts in the requirements because nobody knows it is a requirement: the spreadsheet a team keeps beside the system of record, the email that must be forwarded before a case can close, the senior adjuster everyone asks about the odd ones. You only see it by watching.
OpenAI's own description of early scoping, as reported by the Pragmatic Engineer, begins with "A couple of days onsite with the customer": "Sit with users, map out their processes", "Identify the biggest value areas", "Prototype with synthetic data", then "Whiteboard ideas and prioritize them" (Pragmatic Engineer, T3). Palantir's Deployment Strategist posting asks for the same thing in one sentence: "Go onsite and meet with customer analysts to understand the critical questions they need to answer and locate their biggest pain points" (Palantir posting, T4 for practice, primary for what is asked).
Expect the floor to be messier than the org chart. Sierra, describing its agent strategists, says the work is "mapping the workflow, collecting the right information (standard operating procedures, call transcripts, process documentation, audio calls)", and cites a global travel company with "over 20 internal stakeholders and no APIs" (Sierra, 2026-05-19, T2: vendor-reported). Twenty stakeholders means twenty definitions of better. No APIs means the data question is already a discovery question, not a later one; Data Access and Integration picks it up from there.
The job-post wording quoted in this chapter was re-read on the live postings in October 2026: Palantir's FDSE (US Government) post with the "flights" and "money laundering" questions, Palantir's Deployment Strategist post with "locate their biggest pain points", and Glean's Founding Forward Deployed Engineer post with "Before you write a line of code". Postings change without notice; the argument here does not depend on them.
What to observe and what to ask
Discovery has five jobs, and each has a method that works better than asking.
Shadow the workflow. Sit beside the people who do the work for at least a few hours each, across more than one shift or team if the work varies. Ask them to narrate, and write down every system they touch, every copy and paste, every wait, and every time they ask a colleague. Draw the workflow from what you saw, then show the drawing to the people you watched and let them correct it. A workflow mapped only from interviews is the org chart's version of the work.
Get volumes and time as ranges. Ask "how many a day, on a slow day and a bad day?" and "how long does one take, an easy one and a hard one?" Ranges are honest about what people know, and they are what you need to check whether the lever is big enough to matter. Then check the ranges against something counted: a queue length, a log, your own tally during the visit. If you cannot find any countable source, write that down. It means the baseline has to be measured from scratch in scoping, which changes the plan.
Find who owns the data and the decision. For each system in the workflow, name the person who can grant read access and the person who can approve a change to what the workflow does. They are rarely the same person, and they are rarely the sponsor. Qureshi calls data integration "the core of what the company does", and its first step is "negotiating with 'data owners'"; you want their names in week one, not in week four.
Find what "better" means to each stakeholder, and what each fears. Ask every stakeholder the same two questions: what would make this better for you, and what is the worst thing that could happen if a machine helped with it? The answers will conflict. That is the point: a single stakeholder's definition of better becomes the success metric by default, and the others turn up later as objections. The feared failure is usually the most important line in the guide. It tells you what must never happen, which becomes a hard constraint in the build and a question in the security review.
Spot the gap between the org chart and the real process. Who does the work the process says someone else does? Which step exists on paper and is skipped in practice? Which person is the unofficial router everyone asks? These gaps are where an AI system will fail first, because it will be built to the documented process.
Most of what you hear in discovery falls into a few recurring shapes. The grid below gives the usual meaning and the check to make.
The fifth row deserves a note. Asking experts to judge the same handful of cases is cheap, it surfaces whether "correct" is even agreed, and those cases become the first entries in the eval set that Evals: Proving It Works builds out. Starting the eval in discovery is not premature. It is the cheapest discovery tool there is.
Write the stated and the observed problem side by side
Discovery is finished when one page exists with four boxes on it: the problem as the customer stated it, the problem you observed, the evidence for the observed version, and the business lever it moves. The stated problem stays on the page. Deleting it makes the sponsor defend it; keeping it beside the observed one lets her see the connection and choose.
The evidence box is what makes the page persuasive. It should contain things the sponsor can check without trusting you: a range from the people who do the work, a count from a queue or log, a pattern in call reasons, a quote from someone on the floor. RAND's pricing example is the warning here. The leaders in that example were not wrong about wanting a price; they were wrong about which price. The evidence box is where you show which one.
Then get the sponsor to sign the observed version. The signature matters more than it looks. Scoping will turn the observed problem into a metric, a threshold and a "not now" list, and every one of those will be challenged by someone who preferred the stated problem. A signed page settles that argument once. RAND's recommendation that leaders commit a team to one problem "for at least a year" starts here: the commitment is to the observed problem, not to the request.
If the sponsor will not sign, you have learned something more important than the problem. Either she does not own the lever you found, or she does not believe the evidence, or the engagement was bought for a reason nobody has said aloud. Each of those is a conversation to have now, with the lead, before a line of code. When Engagements Fail returns to what to do if it cannot be resolved.
Template: the discovery interview guide
Use one guide per stakeholder group, and merge them into the summary box at the end. Ranges, not point figures. Mark every number with where it came from: said, counted or measured.
DISCOVERY INTERVIEW GUIDE Engagement: ____________ Interviewer: ____________ Date: ________ Stakeholder (role, team): ______________________________________ 1. BUSINESS LEVER Which number would you most like to move this year? ____________ Who reports it, and to whom? _________________________________ Where do you think it is lost? (their hypothesis) ______________ 2. CURRENT WORKFLOW (observed, step by step) Trigger: ______________________________________________________ Steps (system touched / wait / handoff at each): ______________ Workarounds seen (side sheets, inboxes, "ask X"): ______________ 3. PEOPLE AND HANDOFFS Who does each step: ___________________________________________ Unofficial experts / routers: _________________________________ 4. VOLUMES AND TIME (ranges; source: said / counted / measured) Cases per day (slow day - bad day): ___________________________ Time per case (easy - hard): __________________________________ Share of cases that are hard: _________________________________ 5. DATA Where it lives (system, format): ______________________________ Owner who grants read access: _________________________________ Owner who approves changes to the workflow: ___________________ How to get access, and how long it usually takes: _____________ 6. WHAT "BETTER" MEANS TO THIS STAKEHOLDER ______________________________________________________________ 7. THE FEARED FAILURE Worst thing that could happen if a machine helped: ____________ SUMMARY BOX (complete once, across all interviews) Stated problem: ____________________________________________ Observed problem: ____________________________________________ Evidence: ____________________________________________ Lever it moves: ____________________________________________ Confirmed by sponsor (name, date): ___________________________
Filled from the composite. The ranges are the composite's, not a real insurer's.
DISCOVERY INTERVIEW GUIDE (merged)
Engagement: P&C insurer, 30-day paid pilot Interviewer: FDE
Stakeholders: COO, claims director, adjusters (auto FNOL),
contact-centre head, claims-platform IT, model risk
1. BUSINESS LEVER
Number to move: claims cycle time (COO; reported to the board)
Stated request (IT, via sales): chatbot for policyholders
2. CURRENT WORKFLOW (observed, auto first notice of loss)
Trigger: documents arrive in shared inbox + scanning queue
Steps: open -> identify type -> match to claim -> route to team
-> first action by assigned adjuster
Workarounds: personal tracking sheets; senior adjuster asked
about unclear documents
3. PEOPLE AND HANDOFFS
Adjusters sort their own intake; misroutes bounce between teams
Unofficial router: one or two senior adjusters per team
4. VOLUMES AND TIME
Share of adjuster day spent sorting: about 30-50%
(said, then counted during the visit)
Status calls tied to claims with no first action: most of a
sampled set (counted from call reasons)
Time-to-first-action: not reported; derivable from the week-1
claims-system timestamps (said, then checked)
5. DATA
Claims system (on-prem); document store (separate)
Read-access owner: claims-platform IT lead
Change approver: claims director; model risk for any automation
Access route: replica requested; expected to take weeks
6. WHAT "BETTER" MEANS
COO: shorter cycle time Claims director: fewer misroutes
Adjusters: less sorting Contact centre: fewer status calls
IT: nothing writes to claims Model risk: nothing decides alone
7. THE FEARED FAILURE
An injury or litigation-flagged claim routed to the wrong team
SUMMARY BOX
Stated problem: "A customer chatbot"
Observed problem: FNOL documents wait to be sorted and routed by
hand; claims sit with no first action
Evidence: 30-50% of adjuster day on sorting; status calls
cluster on claims with no first action
Lever it moves: time-to-first-action -> claims cycle time
Not now: chatbot; property claims
Confirmed by sponsor: COO, mid week 2
Three lines in the filled guide carry the next three chapters. "Not reported; derivable from timestamps" is the baseline Scoping and the First Win has to turn into a number. "Replica requested; expected to take weeks" is the opening move of Data Access and Integration. "Nothing decides alone" and the feared failure are the first inputs to Security, Governance and Model Risk, and they will become a rule enforced in code outside the model.
Failure signals
These show up early and are cheap to fix in week one. By week four each of them has turned into rework.
One gap in the evidence should be said plainly. There is no independent study of FDE discovery itself: how long it should take, which methods find the real problem, or how often the observed problem turns out right. Everything in this chapter after RAND is practitioner and vendor testimony, T2 to T4. It agrees, and that is worth something, but it is agreement among people who sell the method. Appendix A grades the chapter's cases.
Ready to move on
Move to scoping when every line below is true. If one is not, the gap goes on the scope's risk list with an owner and a date.
- The observed problem is written down beside the stated one, with evidence, and the sponsor has confirmed the observed version in writing.
- The workflow is mapped from watching it, and the people you watched have corrected the drawing.
- The data owner for each system in the workflow is named, and the first access request has been sent.
- The baseline is measurable: you know what to count, where to count it, and who will let you.
- Each stakeholder group's definition of "better" and its feared failure are recorded, and the conflicts have been shown to the sponsor.
- The original request has a home: inside the observed problem, or on the "not now" list.
The one-page habit. Before scoping starts, put "stated problem" and "observed problem" side by side on a single page, with the evidence underneath, and get the sponsor to sign the observed version. Every later argument about scope, metric and threshold goes back to that page. If you can only do one thing from this chapter, do this.