Confidential Computing & Attested Inference

What a TEE actually proves.

A trusted execution environment is a region of a machine that the machine's own operator cannot read. Three mechanisms make that true, and the third is the one that matters:

• Memory encryption. The CPU encrypts the enclave's RAM with a key held in hardware, so a hypervisor, a host process, or someone with physical access to the DIMM sees ciphertext. On the CPU side this is Intel TDX, AMD SEV-SNP, or AWS Nitro Enclaves; on the accelerator side, NVIDIA's Hopper-generation H100 and H200 GPUs support a confidential-computing mode that encrypts GPU memory and the CPU-to-GPU path, which is what makes this usable for inference rather than just for key handling.
• Measurement. As the environment boots, the hardware hashes what gets loaded — firmware, kernel, the container image, the guest components — into a measurement. Change one byte of the serving code and the measurement changes.
• Attestation. The hardware signs a report containing that measurement with a key rooted in the silicon vendor's certificate chain. A remote verifier checks the signature, compares the measurement against an expected value, and only then releases something — typically the key that decrypts the model weights or the client's data. This is the whole trick: the secret is released to a specific piece of code, not to a machine or an account.

For LLM inference in confidential mode the reported throughput cost is usually in the low single-digit percentages, small enough that performance is rarely the reason teams decline. Availability is: confidential GPU capacity is a subset of a subset, and it is priced accordingly.

One threat removed, four left standing.

The removed threat is precise and worth having: the party operating the infrastructure — a cloud provider, a colocation host, a compromised hypervisor, a co-tenant — can no longer read your prompts or the model's activations out of memory. If your compliance conversation is "we cannot use a hosted GPU because the operator could theoretically read the data", this technology is the answer to that sentence and there is no other.

Everything else survives:

• The application above the enclave. Something logged that prompt. An agent emits spans containing the prompt as actually sent, tool arguments and results — that is the entire point of observability — and those spans land in a normal database with normal access control. An encrypted enclave feeding a plaintext trace store has moved the leak, not closed it. See trace sampling and retention.
• The counterparty you chose. A TEE excludes the operator, not the recipient. If a model vendor runs the enclave, the vendor still defines what the code inside it does; what you gain is the ability to check that the code is the code they published, not permission to stop trusting them. The retention and training-use questions are contractual, and they are covered under data governance, not under silicon.
• The model's behaviour. Confidential computing is orthogonal to whether a prompt injection makes your agent exfiltrate the data itself, through a tool call that is entirely legitimate from the enclave's point of view. Encryption does not care what the plaintext says.
• Metadata. Request volume, timing, sizes and destination are visible to whoever runs the network. For a small number of use cases that side channel is the sensitive part.

An attestation is worth exactly as much as your verification policy.

The measurement is a hash. A hash means nothing until it is compared to an expected value, and the question of who supplies that expected value is where most deployments quietly give the guarantee back.

• Verify it yourself, or you have bought RAM encryption. If your client accepts any well-formed attestation document without checking the measurement against a value you hold, you have confirmed that some genuine enclave is running something. That is a materially weaker statement than the one on the marketing page.
• Ask where the expected measurement comes from. A vendor who runs the enclave and also tells you what its measurement should be has given you a promise with extra steps. The strong version is a published, reproducible build you can measure independently; the workable version is a documented, versioned measurement plus notification before it changes.
• Gate the key, not the log line. Attestation is only load-bearing when something is withheld until it succeeds. The standard pattern is that a key broker releases the data-decryption or weight-decryption key only on a valid report — the same trust-on-first-use-versus-verified split that agent identity and attestation draws one layer up.
• Decide what happens on failure. An attestation check that logs a warning and proceeds is a check that will proceed on the day it matters. Failure has to mean refusal, which means someone has to be willing to take an outage for it.

Three situations where it earns its cost.

Confidential inference is not a general upgrade; it is a specific answer to a specific trust structure, and it costs you capacity, region choice, model choice and operational complexity.

• Regulated data on hosted accelerators. You have no GPUs, the data is health, financial or government-classified, and the blocking objection is the operator. This is the mainstream case and the one the cloud offerings are built for.
• Two parties who will not show each other the data. Two hospitals, an insurer and a bank, a model owner and a data owner. The enclave is the neutral third place, and attestation is what lets each side verify the agreed code is what ran — a structurally different use from the first, and the one where the technology is genuinely irreplaceable.
• Protecting the weights from the person running the machine. The direction most people forget. A vendor shipping a proprietary model into a customer's environment can release the decryption key only to an attested runtime, which is how on-premise deployment of a closed model becomes possible at all.

And the honest negative case: if what you actually want is that the data never leaves your building, the simpler instrument is not to send it. A quantized open-weight model on hardware you own answers the residency question with no attestation chain to verify and no confidential-capacity queue to join — see small & local models and local knowledge bases for what that costs in quality.

Related: data residency & sovereignty for the jurisdictional half of the same question, serving & access for who is on the other side of the API, and sandboxing & code execution for isolation in the other direction — keeping the agent's code away from your machine rather than the machine away from your data.