Building in Their Environment

What this phase is for, and how it fails

By now the engagement has a signed threshold, a data path, a model-risk decision and a climbing eval. It lacks a system that runs where the customer runs things. This phase builds it: on their hosting, behind their identity provider, calling the endpoint they approved, shipped through their change process, and still working after the vendor's people go home.

The FDE does not choose the ground. A product engineer builds on infrastructure their own company picked; an FDE builds on infrastructure somebody else picked years ago, for reasons unrelated to AI. Those constraints are not obstacles to route around. They are the specification.

The phase fails in three ways, and all three come from treating the customer's environment as a deployment detail.

Importing your own stack. The design assumes the services the vendor uses at home: a managed vector database, a hosted orchestration framework, outbound calls to a public endpoint. Then the customer's platform team says none of those exist inside their network, and the design starts over in month two.

Rules in the prompt. The customer has a rule that must never break, and the team writes it into the system prompt in capitals. It holds in the demo and on most of the eval. "Most" is the problem. The first time it breaks in production, the conversation is with the customer's risk function, not with the eval.

Cleverness nobody can operate. The team builds an agent loop with tools, memory and a planner because it is interesting, when a single model call with a fixed output would clear the threshold. It works while the FDE is there to read the traces, and nobody can maintain it the week the FDE leaves.

A word on evidence. This is one of the thinnest chapters in the book: no measured study says which architectures succeed inside customer environments. What exists is secondary reporting on FDE teams (T3), one vendor's engineering guidance and one former FDE's memoir (both T4), and job posts, which show what employers ask for but not how the work goes. The argument rests on those, and on wiki pages for the stack it does not re-teach.

The composite: their VPC, their rules, a boring pipeline

Three choices carry that beat. The platform team's terms became the design brief. The rules left the model for code that can be tested to 100%, and the model kept only the authority to be more careful. The architecture stopped where the eval said it was enough.

Inventory their constraints before you design

The defining fact of the FDE's build work is where the code lives. Gergely Orosz reports that OpenAI's FDEs "write code directly on customer infrastructure and use customer tooling for this". Solutions architects, by contrast, "rarely write code on customers' infrastructure" (Pragmatic Engineer, August 2025; T3, secondary reporting from interviews). Anjor Kanekar, a Palantir FDE for seven years, "was on the final assembly line at aerospace manufacturer Airbus", and the piece notes "that many of his peer FDEs worked in similarly unconventional environments, including airgapped ones" (T3).

Employers write the same expectation into their postings. Sierra's Deployed Infrastructure Engineer needs a "track record of managing deployments in customer-owned cloud environments, including navigating security reviews, compliance requirements, and change management processes" (posting, April 2026; T4, evidence of what the employer asks for, not of how the work goes). The same post wants someone who thinks "about upgrades, rollbacks, monitoring, and failure modes from the start, not as an afterthought". At the far end of the range sits cleared, on-site and air-gapped work, where the constraints are absolute (box below).

Most engagements are not air-gapped, but the lesson transfers: learn what the environment forbids before you draw a box. Ask the platform team, in the first week of the build, for written answers to seven questions:

1. Hosting. Where can a workload run: their cloud account, their container platform, their data centre, a vendor-hosted tenant? Who provisions it, and how long does that take?
2. Model endpoint. Which endpoint did model risk approve, in which region, at which version? Who is told when it changes? If the approved option is a model they host themselves, the wiki's self-hosted inference page covers what that adds to the build.
3. Identity. Which identity provider do services and people authenticate through, and who issues a service identity? A shared vendor key is the wrong answer.
4. Network. Is there outbound internet access? Is there a proxy, an allow-list, a package mirror? Can your build pull dependencies at all?
5. Data stores. Which stores and queues already exist that you are allowed to use? A new database is a new thing they must back up, patch and audit.
6. Change process. How does a change reach production: a pipeline, a change board, a release window? How long from merge to running?
7. Operations. Who will be paged when it breaks, and what tools do they already watch?

Write the answers into the context section of the decision record (10.7) before you write its decision. Every answer removes options, and that is the point: a design that respects all seven is usually smaller than the one you would have drawn alone, and it is the only one that survives the change board. Where the system must run untrusted inputs or code, the wiki's sandbox and isolation patterns cover the options; the customer's platform team decides which of them they will run.

Put the rules that must hold outside the model

Every engagement turns up a few rules that must never be broken. A litigation-flagged claim never goes to the fast queue. A payment above a limit always needs a second approver. A critical component always has at least two suppliers. The customer will say these rules out loud in discovery, and scoping often writes them down as guardrails. Before the build, get the full list from the threshold owner with one question: "Which wrong answer would you have to explain to a regulator, a court or the board?" Write each answer as one sentence with a trigger, an action and an owner. The build then decides where each one lives.

The best-documented field example is an OpenAI FDE engagement with an automotive manufacturer in Asia-Pacific. ZenML's LLMOps database summarises it from an interview with Colin Jarvis, who leads the team (ZenML, 2025; T3, a secondary summary of the vendor's own account, with no outcome metric reported). The team built a supply-chain agent on one principle: "use determinism wherever possible and LLMs only where their probabilistic nature adds value." Hard constraints such as "always maintaining at least two suppliers for critical components like tires" were "checked 100% of the time through deterministic code rather than trusting the LLM." The model did not optimise the plan directly either; it was "given access to a simulator" and explored options, while code enforced the limits. One case, secondhand, with no result attached: treat it as a pattern worth copying, not as proof that it works.

The reasoning does not need the case, though. A model's output is a distribution, and the best eval result is still a percentage. A rule the customer calls "never" cannot be satisfied by a percentage, so it has to sit in code that runs on every request and does the same thing every time. In practice that code goes in three places.

• Before the model. Facts the system already knows decide the route without asking the model. If the claims system has set the litigation flag, the claim goes to the litigation queue, and the model's opinion is not requested or is ignored.
• After the model. The output is parsed against a fixed schema, and only values from an allowed list survive. A queue name the rules forbid for this claim type is replaced by the human queue. A malformed answer is treated as no answer.
• Around the action. The model's authority stops where the rule begins. In the composite the agent only proposes; an adjuster acts. The wiki's human-in-the-loop page covers where approvals sit, and policy-as-code for agents covers rule engines for when the rules outgrow a few dozen lines.

The hard case is a rule whose trigger only the model can see. A litigation flag is a field; an attorney's letter in an unflagged claim's email thread is not. Code cannot read the letter, but it can limit what the model's reading is allowed to do. Give the model authority in one direction only. It may move a case toward caution, such as sending a claim to a human because a document mentions counsel. It may never move a case away from caution: it cannot clear a flag or send a held claim down the fast path. A false alarm costs an adjuster a minute; a miss is what the rule exists to prevent. Write the permitted direction beside the rule in the decision record.

Test the rules separately from the eval. The eval reports routing accuracy as a percentage and is allowed to be below 100%. The rule tests are ordinary unit tests: one case per rule, plus the edge cases the customer's experts name, and the build fails if any of them fails. Reporting them inside the eval percentage hides the one failure that matters among the hundreds that do not.

This is also what keeps the security review's decision standing (see the security chapter). A rule in code is something a reviewer can read, test and sign. A rule in a prompt asks the reviewer to trust a distribution, and a careful reviewer will not.

Build the least clever thing that clears the eval

Once the rules are out, what is left for the model is usually smaller than expected: read a messy document, choose from a short list, give a reason. That job rarely needs an agent.

Anthropic's engineering guidance, drawn from work with dozens of teams building "LLM agents across industries", says so: "we recommend finding the simplest solution possible, and only increasing complexity when needed" (Anthropic, December 2024; T4, a vendor's guidance from its own customer work, not a measured comparison). It suggests developers "start by using LLM APIs directly: many patterns can be implemented in a few lines of code." Frameworks, it warns, "often create extra layers of abstraction that can obscure the underlying prompts and responses, making them harder to debug." And it names the cost of autonomy: "The autonomous nature of agents means higher costs, and the potential for compounding errors."

An FDE has an extra reason to take that advice. Every layer you add is a layer the customer's team must understand after you leave, inside an environment where they cannot install your favourite debugging tool. Nabeel Qureshi's memoir of Palantir makes the point from the other side. His first real customer engagement was Airbus, where he "moved out to Toulouse for a year and worked in the factory alongside the manufacturing people four days a week". The software pulled "disparate sources of data — work orders, missing parts, quality issues" into one place. He is candid about how ordinary that was: "These are all sort of basic software things", yet "just deploying these ‘best practice’ UIs to the real world is insanely powerful" (Qureshi, October 2024; T4). His "4x" for the A350 manufacturing pace is a former employee's recollection with no method; do not repeat it as a result. The admission is the useful part: the value came from basic software fitted to the real workflow.

Use a ladder, and climb a rung only when the eval tells you to:

1. Rules alone. If code can decide a share of the cases from facts already in the system, let it, and send only the rest to the model.
2. One model call with a fixed output. A prompt, a few examples, a schema. Most classification, extraction and routing work stops here.
3. A fixed chain. Several calls in a set order, each checkable on its own: extract, then classify, then check. Add a step when the eval's failure categories show one stage failing.
4. An agent loop. The model chooses tools and steps. Reach for it only when the path through the task genuinely cannot be known in advance, and expect the eval, the audit log and the handoff to get harder.

The wiki's agents vs chatbots vs workflows page draws the same lines. The test for climbing is the failure categories from the evals chapter, not curiosity. If the routing errors cluster in documents the extraction mangles, fix the extraction; a planner will not help. If the customer already exposes the systems you need as tools, an MCP server can be the cleanest interface (what is MCP), but only on a rung where the model genuinely calls tools.

One caution. "Least clever" means least clever that clears the eval. A design too simple to reach the threshold is not modest; it is a failed engagement with tidy code. The ladder exists so that you climb when the numbers say so and not before.

Build it so they can run it

Qureshi is honest about the habit this phase must break: "FDEs tend to write code that gets the job done fast, which usually means – politely – technical debt and hacky workarounds" (T4). In a customer's environment, that debt becomes theirs the day you leave. Five habits keep it small.

Ship through their pipeline from the first week. If go-live is the first time the change board sees the system, you will spend the go-live window in their queue. Push something trivial through their full path early. Test in a staging environment inside their network that mirrors production; the wiki's staging environments for agents page covers what that environment needs.

Pin every version, and log it. The model endpoint, the prompt, the examples, the rules and the code each get a version, and every suggestion's audit entry records all of them. When the platform team upgrades the endpoint, rerun the eval and the rule tests before the change reaches production. A routing result you cannot reproduce cannot be defended to model risk.

Fail closed, to the old process. When anything goes wrong (the endpoint times out, the output fails the schema, a rule rejects it), the case goes where it went before the engagement. The customer's operations already know how to handle that path. Failing to the old process turns an outage into a slower day instead of an incident.

Use what they already run. Their logging, their monitoring, their secrets store, their queue. A component they do not already operate is one more thing to hand over, and one more reason for the platform team to say no.

Re-measure on the production stack. The eval number from the pilot was measured somewhere else. Have the threshold owner's team run the held-out set through the production deployment, inside their environment, and confirm it matches before anyone talks about go-live. Differences in text extraction, endpoint versions or timeouts show up here, not in the demo.

None of this is new to an infrastructure engineer. What is particular to the FDE is doing it on ground you do not control, for a team that will own it after you. The next chapter turns that ownership into an acceptance test.

The template: the architecture decision record

One record per significant decision, written before the build commits to it and signed by the people who will live with it. The customer's platform owner signs that it fits their environment; model risk signs that the rules are where they approved them; the FDE lead signs that it can clear the eval. Keep it in the customer's repository, not the vendor's.

ARCHITECTURE DECISION RECORD                ADR-__  date: ________
Status: proposed | accepted | superseded by ADR-__

1. CONTEXT (their constraints, in writing from their platform team)
   Hosting:          _______________________________________________
   Model endpoint:   ____________  region: ________  version: ______
   Identity:         _______________________________________________
   Network/egress:   _______________________________________________
   Allowed stores:   _______________________________________________
   Change process:   _______________________________________________
   On-call owner:    _______________________________________________

2. DECISION (one paragraph)
   ______________________________________________________________

3. RULES THAT MUST HOLD (never decided by the model)
   Rule                       Enforced where      Test
   - ______________________  before | after    ____________
   - ______________________  before | after    ____________
   Rule tests must pass: 100%, on every change
   Model may move a case toward caution only: Y / N  (which rules)

4. WHAT THE MODEL DECIDES
   Input:   ______________   Output schema: ______________________
   Allowed values: _______________________________________________
   Authority: propose only | act with approval | act

5. OPTIONS CONSIDERED
   Option                     Rejected because
   - ______________________  ___________________________________
   - ______________________  ___________________________________

6. FAILURE BEHAVIOUR
   Endpoint down / bad output / rule rejects  ->  ________________

7. VERSIONS PINNED AND LOGGED PER DECISION
   model | prompt | examples | rules | code

8. EVAL GATE
   Held-out set re-run on this stack: ______  matches pilot: Y / N
   Climb to a more complex design only if: _______________________

9. REVERSIBILITY
   How to turn it off: ___________________   Who can: ____________

10. SIGN-OFF
   Customer platform owner (fits our environment): ______  date: __
   Model risk (rules where approved):              ______  date: __
   Vendor FDE lead (can clear the eval):           ______  date: __

Here it is filled from the composite. Every figure is a range, as the composite rules require; a real record would carry exact versions and names.

ARCHITECTURE DECISION RECORD                ADR-03  date: month 2
Status: accepted

1. CONTEXT (their constraints, in writing from their platform team)
   Hosting:          insurer's container platform, inside their VPC
   Model endpoint:   approved by model risk  region: in-region
                     version: pinned; change notice from platform
   Identity:         insurer's identity provider; read-only service
                     identity issued by IT security
   Network/egress:   no outbound internet; internal package mirror
   Allowed stores:   side table owned by claims-platform IT; existing logs
   Change process:   weekly change board, via insurer's pipeline
   On-call owner:    insurer platform ops (runtime, after handoff);
                     claims-platform IT (connector, side table)

2. DECISION (one paragraph)
   Route auto FNOL packets with one model call returning a queue
   and a reason in a fixed schema. Deterministic rules run before
   and after the call. Suggestions go to the side table only; no
   agent loop, no retrieval index, no new datastore.

3. RULES THAT MUST HOLD (never decided by the model)
   Rule                       Enforced where      Test
   - litigation flag set =>   before + after      unit tests,
     never the fast queue                         adjuster edge cases
   - listed claim types =>    before              unit tests
     always a human queue
   - queue not in allowed     after               schema tests
     list => human queue
   - counsel named in packet  after               unit tests,
     (model hint) => human                        attorney-letter case
   Rule tests must pass: 100%, on every change
   Model may move a case toward caution only: Y  (litigation: may
   raise a hint; can never clear a flag or pick the fast queue)

4. WHAT THE MODEL DECIDES
   Input:   extracted packet text   Output schema: queue + reason
   Allowed values: the auto triage queues the claims director named
   Authority: propose only (adjuster acts)

5. OPTIONS CONSIDERED
   Option                     Rejected because
   - vendor-hosted service    data must stay in insurer VPC
   - agent loop + framework   eval failures were extraction and
                              examples, not planning; harder to run
   - managed vector store     not on platform list; not needed

6. FAILURE BEHAVIOUR
   Endpoint down / bad output / rule rejects  ->  default human
                                                   queue (old path)

7. VERSIONS PINNED AND LOGGED PER DECISION
   model | prompt | examples | rules | code  (in every audit entry)

8. EVAL GATE
   Held-out set re-run on this stack: month 2  matches pilot: Y
   Climb to a more complex design only if: one failure category
   persists that a single call cannot fix

9. REVERSIBILITY
   How to turn it off: config flag hides suggestions  Who can:
   claims ops lead or platform on-call

10. SIGN-OFF
   Customer platform owner (fits our environment): yes   date: m2
   Model risk (rules where approved):              yes   date: m2
   Vendor FDE lead (can clear the eval):           yes   date: m2

Two lines in the filled version do most of the work. Section 3 puts each "never" in code with its own test, so the eval percentage never has to carry it, and it says in one line how far the model may push a case it can only see in the documents. Section 5 records why the cleverer options were rejected, so the next engineer, who may be the customer's, does not add them back on a slow afternoon.

Failure signals

Each of these shows up during the build, usually before anyone outside the team notices, and each points to one fix.

Ready to move on

Move into production and handoff when every box is ticked:

• The system runs inside the customer's environment, deployed through their pipeline under an identity their IT security issued.
• Every rule that must hold is enforced in code outside the model, with tests that pass on every change; where only the model can see a rule's trigger, it can push cases toward caution and never away.
• The architecture decision record is signed by the customer's platform owner, model risk and the FDE lead.
• Every version is pinned and logged with each decision, and the fallback to the old process has been tested.
• The held-out set, re-run on the production stack, matches the pilot result, and the threshold decides go-live.