Feeding the Product

What this phase is for, and how it fails

By the end of production and handoff, the customer has a working system and an operations team that can run it without you. For the customer, the engagement is done. For your employer, half of it is. A product company that sends engineers into customer buildings for months is doing services work, and the only reason to do it is that the work comes back as product. If it does not, the vendor has built a consultancy with worse margins.

Colin Jarvis, who runs forward deployed engineering at OpenAI, puts the choice bluntly. Before you build an FDE team, decide whether you are "relying on the services revenue as like a key revenue stream", because that is "a very different motion" from betting on "successful product bets that are going to come out of these FDE engagements". He has watched the failure from the inside: "from working in consulting in the past… I always saw a failure mode in those consulting firms where they had a vision of like we're eventually going to be a product company, but unfortunately… the short-term lure of services revenue just starts to drag everyone in that direction" (Altimeter interview, November 2025; T4, a vendor executive describing his own organisation, quoted from the auto-generated transcript with filler words cut).

That is the chapter's argument. The loop back to product is what separates an FDE from a contractor, and the evidence that employers believe it is in every lab and platform posting this book checked. The loop fails in two opposite ways.

Nothing comes back. The FDE ships, hands off and moves to the next customer, and the second customer gets the same build from scratch. Pragmatic Engineer spoke to engineers who took FDE jobs and were disappointed by exactly this. Its summary: "They don't contribute back into the platform", and they do little beyond integrating what the product team built. One of them said, "This FDE job was a typical IT services mindset… The company wanted to use me more on the engagement lead side, and nothing on software development," and "I quit 4 weeks later" (Pragmatic Engineer, March 2026; T4, two anonymous engineers, one quoted). Two people are not a market, but the complaint names the failure precisely. When the loop is missing, good engineers notice first, and they leave.

Too much goes back, too soon. The opposite failure is turning one customer's build into a platform before anyone knows which parts are general. Jarvis calls it the biggest mistake his team made: "one of the biggest ones is probably generalizing too early", building a general solution and then going "looking for a problem" for it (same interview, T4). A product built from one customer is that customer's product with the names filed off.

Be clear about the evidence: every account here comes from a vendor describing its own loop, a former employee, a secondary write-up or a job posting. Nothing is T1. Nobody has independently measured whether field feedback improves products, or how much FDE work is reused, and the numbers you will see are targets. What the evidence does show is that the employers who hire for this role all say they pay for the loop, which makes it part of the job whether or not anyone has measured it.

The composite: the second insurer

The rest of the chapter takes that beat apart: evidence logged during the engagement, a line between the pattern and the customer, and a template that was not a product until a second customer used it.

Keep a field log from week one

A product team will not act on "customers want better document handling". It will act on "this failure cost three weeks below threshold at one insurer, here are the categories, and here is the next customer who will hit it". The second sentence can only be written by someone who kept notes while it was happening. By the time you write the brief, memory has rounded every number and smoothed every workaround.

Keep a field log, separate from the customer's project tracker, which belongs to them. Four sources feed it, and most of them already exist if you followed the earlier chapters:

• Eval failure categories. The evals chapter has you sort failures into categories and fix them one at a time. Each category is a candidate finding about the model or the platform. Log the category, how often it occurred, and how long it took to fix.
• Workarounds. Every time you built around the platform (a parser it lacked, a retry loop, a rule moved into code because the model could not hold it), log what you built and why.
• Overrides. In production, the users' corrections are a running record of where the system is wrong. Log the patterns, not the records.
• Second builds. When you build something you have built before for another customer, log it twice.

Keep each entry to one line with six fields: the date; the source (eval category, workaround, override pattern or second build); what happened, stripped of customer identifiers; the cost, in FDE-days or weeks below threshold; the destination; and whether any of it belongs to the customer. Ten minutes on a Friday is enough. The cost field is the one the brief will need and the one memory loses first.

Each finding has one of three destinations, and the log should say which.

Product takes components and features: the thing you built that the next customer will also need. Research takes model weaknesses: a class of task the model gets wrong, described precisely enough to collect training or evaluation data for. The playbook takes deployment patterns that are not code at all: the order of the security review, the shape of the data-access request, the handoff drill.

The rule for choosing is what the fix was. If you wrote code around the platform, the finding goes to product. If you changed a prompt or a rule and the model still misses a named class of input, it goes to research. If nothing was built and the time went on sequence, people or paperwork, it goes to the playbook. One finding can go two ways: in the composite, the mixed-packet failures went to research and the pipeline built around them went to product.

FDEs most often miss the research destination, though it has the clearest examples. Gregor Ojstersek's account of a conversation with Jarvis describes a code-migration engagement for a customer in Japan. When the model kept making simple mistakes, the FDEs did not pass the raw complaint upstream. They reported "The model struggles with these types of COBOL-to-Java tasks" and "We need more training data around these examples" (Engineering Leadership, August 2026; T3, a secondary account with no outcome reported). Pragmatic Engineer describes a voice engagement for call-centre automation in which FDEs "created evals on the voice model", took the data to OpenAI's research department, and the lab's real-time voice API "was also improved by this work" (Pragmatic Engineer, August 2025; T3). Neither account says how much better the model got. Both show the shape of a useful research finding: a named class of task, evidence from a real deployment, and a request a research team can act on.

OpenAI's enterprise guide names the mechanism: "We use iterative deployment to learn quickly from customer use cases and use that information to accelerate product improvements" (AI in the Enterprise, 2025; T4, a vendor describing its own method). Without a field log, iterative deployment is just deployment.

Separate the pattern from the customer

During the engagement, overfit. The customer is paying for their problem, not yours, and an FDE who builds for an imagined second customer while the first one waits is failing both. Nabeel Qureshi, at Palantir from 2015 to 2023, describes the division of labour that made this safe there: "Your job was to solve the problem, and not worry about overfitting; PD's job was to take whatever you'd built and generalize it, with the goal of selling it elsewhere." Field engineers "had to do a bunch of cruft work manually, and PD engineers built tools that automated the cruft work", and he names several Foundry tools that began that way (Reflections on Palantir, October 2024; T4, one former employee's recollection). He calls the combination "One of the key 'secrets' of the company".

A secondary write-up of a podcast with Bob McGrew puts the split as a metaphor: FDEs build "gravel roads" for one customer, which product teams turn into "paved superhighways", and that step is how a company avoids the "consulting trap" (StartupHub, September 2025; T3). The phrases are the writer's, not quoted from McGrew, though they circulate as his. Use the metaphor, not the attribution.

What both versions leave out is the line between the pattern and the customer. Draw it before you write a word of the brief, because a feedback brief that carries the customer's assets is a contract problem, not a product idea. Some things are yours to generalise. Others never leave the building:

• The pattern travels: the shape of the pipeline, the structure of the eval harness, the decision to enforce hard rules in code, the integration approach, the order of the work.
• The customer stays: their data and documents, their labelled held-out set, their business rules and thresholds, their queue names and system identifiers, and anything their contract says they own.
• Failure patterns travel only as descriptions: "mixed packets with a handwritten note" is a finding; the packets themselves are the customer's.

The scoping chapter made the customer the owner of the held-out set so that a pass would be their finding, not the vendor's claim. The same ownership applies here. The second customer gets the harness, never the first customer's test. Their experts label their own, which is also why their threshold means something. When unsure where the line falls, ask the lead and read the contract; do not ask the product team, which wants the answer to be yes.

Let the second customer decide what is general

Jarvis's answer to "generalizing too early" is the most practical advice in the public record on this phase: go deep on one customer, and let the general thing show itself. "Whenever we've just gone super deep on the customer's problem and just been like, 'We're just going to solve this problem.' Almost every time, there's been some kind of generalizable thing… that's popped at the end" (Altimeter interview, T4). The first customer tells you what works. Only the second tells you which part of it is general.

He gives a target for how that compounds: "if the first thing was maybe like 20% reusable, the next two get us to maybe 50% reusable, and then we almost want to like push it into the scaled part of the business" (same interview, T4, a stated target). In the next breath he says, "we're like very much at the start of that journey". They are a goal from one team at one company, not a measurement, and a ZenML summary of a Jarvis interview repeats them (ZenML, 2025; T3). That is an echo of the same speaker, not a second source. They are useful as a shape: low reuse from the first customer is normal, and reuse that has not risen by the third is a signal.

Turn the shape into a rule. When one customer needs a thing, build it in the engagement and log it. When a second engagement needs it, ask for a template, as the composite did. When a third uses the template with configuration changes only, ask for a product, with all three as evidence. If the third needs changes to the template's core, the general part is smaller than you thought: shrink the template rather than grow it.

The clearest public story of the loop working is the one Jarvis tells about customer service. At Klarna in 2023, writing a prompt for each of hundreds of policies did not scale, so the team parameterised instructions and tools and wrapped each intent with evals, scaling from about 20 policies to about 400. They turned the method into an internal framework and open-sourced it. A later engagement at T-Mobile, which he describes as "maybe 10x more complex" in volume and policy count, found the framework "worked like pretty well there with a few… extensions". With production customers behind it, the team went to product, and OpenAI stood up a team to build what became its agent SDK (Altimeter interview, T2). That is the second-customer rule in action: the framework became a product only after a second, harder customer used it with small changes.

Flag the attribution before you tell this story to a customer. Jarvis himself says it "came… from that was a solution architect engagement but very much an FDE style of delivery". Pragmatic Engineer says OpenAI's Solutions Architecture team created the framework (Pragmatic Engineer, T3). The best-known example of FDE work becoming product was not, strictly, an FDE engagement. It still shows the loop, but it shows that the loop belongs to whoever is in the field and owns the outcome, not to a job title.

Get it adopted, not just heard

A brief that nobody acts on is a diary entry. The loop needs someone on the receiving end with the authority and the capacity to build, and that is not something an FDE controls. It is still something an FDE can test for, and route around.

Jarvis is candid that his own loop once had no receiver. Until late 2024, he says, the company's focus was elsewhere and enterprise frameworks drew "not a lot of interest": "we shipped Morgan Stanley, we shipped Klarna… but… there just like wasn't the investment from the business and… our words were falling on deaf ears" (Altimeter interview, T4). That lack of interest was one reason the framework was open-sourced. The company swung back to enterprise at the end of 2024. What he took to product then was evidence from outside: production customers using the primitives, and the open-source version "getting like a lot of stars on GitHub", which he says carried "quite a lot of influence internally". The lesson for an FDE without a VP's ear is that usage beats argument. A second customer using your component is a stronger brief than a well-written one.

When the receiver exists, the loop can be fast. Usman Nasir, a vice president on Salesforce's Forward Deployed Engineering team when the post appeared, says on the company's blog: "I've seen where features have been moved up six months because of the urgency of what we're seeing in the field" (Salesforce, November 2025; T4, a recruiting post on the employer's own blog). Treat the six months as one executive's anecdote, not a rate. The useful part is the mechanism: what moved the roadmap was urgency seen in the field, evidence about customers, not a request from an engineer.

Four habits make adoption more likely:

1. Name the receiver. A brief addressed to "product" goes nowhere. Address it to a person who owns a roadmap, and ask for a decision by a date: build, defer, or decline.
2. Ask for the smallest version. A template, a configuration option, a documented pattern. The composite's product lead said yes to a template and would have said no to a product.
3. Price the cost of not doing it. Say how long the next engagement will spend rebuilding the same thing. Give them a number, as a range if that is all you have.
4. Bring the second customer. If a named upcoming engagement will need the component, say so. It turns a hypothesis into a delivery date.

The FDE's time for this has to come from somewhere. Jarvis describes OpenAI splitting FDE capacity between engagements chosen to prove "definite product hypotheses" and engagements chosen for hard problems that might teach research something (Altimeter interview, T4); Leading FDEs, in Part IV, covers how managers run that split. For the individual FDE, the move is to agree two to three days for the brief with the lead before the engagement starts. A brief squeezed between the next customer's meetings does not get written.

The template: the product-feedback brief

One finding per brief. A brief with five asks gets the easiest one done and the rest forgotten. Write it from the field log, have the lead check the customer line, and send it to a named person.

PRODUCT-FEEDBACK BRIEF                           version / date: ____

1. FINDING (one line)
   ________________________________________________________________
   Destination:  product | research | playbook

2. FIELD EVIDENCE
   Engagements where seen:  ______________________________________
   How it showed up:        eval category | workaround | overrides
                            | built twice
   Frequency / cost:        ______________________________________
   Evidence tier:           measured here | observed | anecdote

3. WHAT WE BUILT IN THE FIELD
   Description:  __________________________________________________
   Where it lives / who owns it now: ______________________________

4. THE LINE
   Generalises (pattern):       __________________________________
   Stays with the customer:     data, labelled sets, rules,
                                identifiers, contract-owned items:
                                __________________________________
   Checked by lead / contract:  __________   date: ______

5. THE ASK (smallest useful version)
   ________________________________________________________________
   Not asking for: ________________________________________________

6. SECOND-CUSTOMER TEST
   Next engagement that needs it:  ________________   starts: _____
   What it would reuse:            ________________________________

7. COST OF NOT DOING IT
   Rebuild time per engagement:  ________ (range)
   Risk if rebuilt each time:    ________________________________

8. DECISION
   Receiver (owns a roadmap):  ______________   decide by: ________
   Outcome: build | defer (until ____) | decline (reason: ______)

Here it is filled from the composite, in month five. The figures are ranges, as the composite rules require; a real brief would carry the logged numbers.

PRODUCT-FEEDBACK BRIEF: claims document triage   v1 / month 5

1. FINDING (one line)
   FNOL packet triage was built from parts; the next insurer will
   need the same pipeline and eval harness.
   Destination:  product (pipeline, harness)
                 + research (mixed-packet failures, sent separately)

2. FIELD EVIDENCE
   Engagements where seen:  1 in production (auto FNOL triage);
                            1 insurer pilot starting months 6-8
   How it showed up:        workaround (packet split, doc typing);
                            eval categories; built from parts
   Frequency / cost:        pipeline + harness took several weeks
                            of a single FDE's time
   Evidence tier:           measured here (eval log, shadow-run)

3. WHAT WE BUILT IN THE FIELD
   Description:  split packet -> classify each document -> propose
                 queue; hard rules enforced in code outside the
                 model; suggestions written to a side table
   Where it lives / who owns it now: insurer's VPC; insurer ops

4. THE LINE
   Generalises (pattern):       pipeline shape, harness structure,
                                rules-outside-the-model, side table
   Stays with the customer:     claim documents, held-out set and
                                labels, routing table, queue codes,
                                litigation-flag rules, threshold
   Checked by lead / contract:  the lead   date: month 5

5. THE ASK (smallest useful version)
   A template: pipeline with routing table, queue codes and hard
   rules as configuration; eval harness with empty case folders.
   Not asking for: a claims product, a UI, a model change

6. SECOND-CUSTOMER TEST
   Next engagement that needs it:  second insurer   starts: month 6-8
   What it would reuse:            skeleton, harness, rule pattern;
                                   NOT data, labels or rules

7. COST OF NOT DOING IT
   Rebuild time per engagement:  a few weeks (range from field log)
   Risk if rebuilt each time:    hard rules re-implemented ad hoc;
                                 each harness built differently

8. DECISION
   Receiver (owns a roadmap):  platform product lead   decide by: m6
   Outcome: build (template only, one engineer, a couple
           of months)

Two lines do most of the work. Section 4 names what stays with the insurer specifically, so the lead can sign it in minutes instead of sending it to legal. Section 5 says what is not being asked for, which tells a product lead the request is small enough to say yes to.

Failure signals

These show up after go-live, usually when the FDE is already on the next engagement, which is why they are easy to miss.

Ready to move on

The engagement is finished for the vendor, not just the customer, when every box is ticked:

• A field log exists, with eval failure categories, workarounds, override patterns and second builds, each tagged product, research or playbook.
• At least one product-feedback brief has been written from it, with one finding, its evidence and its tier.
• The brief lists what stays with the customer, and the lead has checked it against the contract.
• The brief is addressed to a named person who owns a roadmap, with a decision date.
• A second customer has been named as the test of whether the pattern is general.
• Research findings have been sent as failure descriptions, without customer records.

The next chapter, when engagements fail, covers what happens when the insurer's expansion to property claims does not go to plan.