Concepts
AI & agentic AI explained — plain-language entries for newcomers and intermediates.
New to agentic AI? Read these five concepts in order:
Or follow the full guided path in the Field Guide →
AI Foundations
- What is AI, ML & Deep Learning?AI, machine learning, and deep learning are nested circles, not synonyms — and which one you are looking at predicts how a system fails.
- Neural Networks, IntuitivelyA neural network is a big stack of adjustable knobs that turns numbers into numbers; learning is just nudging those knobs toward less error.
- What Is a Large Language Model?An LLM is a huge next-token predictor; scale turned that one simple objective into abilities nobody explicitly programmed.
- Training vs InferenceTraining builds the model’s frozen weights once; inference runs them per request and never changes them — which answers most cost and privacy questions.
- Tokens & TokenizationModels see integers, not text; the hidden tokenization step explains your bill, your context limit, and odd failure modes.
- Embeddings: Meaning as GeometryAn embedding is not a measurement of similarity but a trained opinion about what similarity means, frozen into every vector you store — which is why a question and its own answer land far apart under a sentence-similarity model and close together under a retrieval one, and why Cohere makes you pass input_type, Gemini task_type, and nomic a literal text prefix. Get that string wrong and nothing errors. Covers the geometry, the asymmetry, and why the stored vector commits you to one model, version, width and role convention at once.
- Transformers, at a High LevelSelf-attention lets every token look at every other token; that one idea fixed long-range memory and unlocked GPU-scale training.
- Generation & Sampling: TemperatureThe model returns a probability distribution, not an answer; temperature reshapes it, and temperature 0 is low-variance, not deterministic.
- Hallucination & GroundingA model invents fluent answers because fluency, not truth, is what it optimizes — so fabrication is structural, not a bug awaiting a patch. Grounding puts the evidence in the context and requires the answer to come from it, which does not eliminate invention but makes it cheap to catch.
- Synthetic DataModel-generated training data is safe in proportion to how good your filter is — collapse is a property of the pipeline, not of the data.
- Distillation & QuantizationDistillation trains a new, smaller model; quantization stores the same weights at lower precision. One is GPU-days and irreversible, the other is minutes and is not.
- Post-Training: Base Model to AssistantRefusals, sycophancy, formatting habits and the assistant persona itself are installed after pre-training, in a comparatively cheap layer rebuilt for every release — which is why a minor version bump can move behavior more than a capability upgrade.
- Prefill, Decode & the KV CacheOne model call is two machines with opposite bottlenecks: prefill reads the prompt in parallel and is compute-bound, decode writes one token at a time and is memory-bandwidth-bound. That split explains time-to-first-token, why output costs more than input, and why prompt caching must be a prefix match.
- Reproducibility & NondeterminismTemperature 0 is a sampling rule, not a guarantee: inference servers batch your request with other people’s, and many kernels change their reduction strategy with batch size, so identical greedy requests return different text. The consequence for agents is that a failing run cannot be reproduced by re-running it — reproducibility has to come from records.
- Knowledge Cutoffs & the Missing ClockThe cutoff is a gradient, not a wall — knowledge of the months just before it is thinner than knowledge of two years earlier, which is exactly where confidence outruns evidence. And a model with no clock turns every date question into a fluent, self-consistent, wrong answer.
- Speculative DecodingThe one speedup that provably cannot change what the model says — a cheap draft guesses ahead and the real model verifies — which is why it costs you no eval cycle, and why it quietly loses throughput on a busy GPU.
- Mixture of ExpertsParameter count stopped meaning cost: MoE splits the bill so compute follows the parameters that fire per token while memory follows all of them — which makes the same model a bargain on a rented API and an expensive mistake on your own GPUs.
- Parameter-Efficient Fine-TuningLoRA won because the specialisation ships as a 50 MB file, so one base serves a hundred variants and switching becomes routing — and because rank is a capacity dial that decides whether you can teach behaviour (yes) or facts (no, that is retrieval).
- Constrained DecodingMasking the tokens that would break your schema guarantees the bytes parse and nothing else — and it deletes the model's ability to abstain, so a required field gets filled from an input that never supported it. Put the escape hatch inside the schema, order fields so evidence precedes conclusions, and keep the reasoning outside the grammar.
- Scaling LawsA scaling law is a curve fitted to pretraining loss — startlingly accurate about that one quantity, silent about whether your agent finishes a forty-step task. The 2022 Chinchilla correction moved the compute-optimal ratio by more than tenfold, which is how much a “law” can move; and the axes that now set your bill are post-training and inference-time compute, not parameter count.
- Interpretability & ProbesInterpretability shipped as a production control in January 2026, and what shipped was the least glamorous artifact the field had: a linear probe on one layer's activations, running at roughly 1/50th the cost of an LLM classifier and beating it on accuracy. It reads a signal no text filter can see, it falls apart under distribution shift, and it needs the weights — so for most teams it is a procurement question rather than a project.
Agentic AI
- What Is an AI Agent?An agent is a model placed in a loop with tools, choosing each next action toward a goal — the core mental model.
- The Agent LoopReason → act → observe → repeat: tracing a tool call from the model through the harness into the environment and back.
- Autonomy LevelsA five-rung ladder from suggest to fully autonomous, and why the right level is a per-action engineering decision.
- Agents vs Chatbots vs WorkflowsOne question — who decides the next step — sorts any LLM system into chatbot, pipeline, workflow, or agent.
- Tools, Actions & EnvironmentsWhat a tool really is, read vs write actions, and why the environment — not the model — is where agents become dangerous.
- Goals, Planning & TerminationThe planning spectrum from reactive to deliberative, and the under-appreciated hard problem of knowing when an agent is done.
- When to Use an AgentThe three properties a task needs to justify an agent, the cheaper patterns that solve most cases, and clear do-not cases.
- Risks & Limits of AgentsThe four characteristic loop failure modes, the security shift autonomy brings, and what "safe agent" honestly means.
- Prompt injection, in plain wordsWhat prompt injection actually is, why it's not a bug a vendor can patch, and the three real defenses available to you.
- Agent Memory: Short-Term vs Long-TermThe context window is an agent’s short-term memory and it resets every session; durable behavior needs an external long-term store you write to and retrieve from on purpose.
- Computer Use & GUI AgentsWhen there is no API, an agent can drive the screen itself — reading a screenshot and synthesizing clicks and keystrokes — which unlocks any software but is slow, brittle, and a fresh attack surface.
- Multi-Agent SystemsOne strong agent is the baseline, not the goal; you reach for multiple cooperating agents only when a task is genuinely parallel or needs separate specialized contexts — and you pay in coordination cost, error propagation, and tokens.
- Voice & Realtime AgentsTalking to an agent in real time makes latency the whole design problem, and forces one architectural choice: a swappable STT→LLM→TTS cascade you can inspect, or a single speech-to-speech model that trades that control for speed and natural prosody.
- Human-in-the-LoopHuman-in-the-loop is not the opposite of automation — it is where you place a human checkpoint. Gate the few consequential, irreversible actions and let the rest run; the trap is the rubber-stamped approval that adds latency and false confidence while catching nothing.
- Sandboxing & Code ExecutionCode is the universal tool — and agent-written code is untrusted code, always, because its output depends on inputs you do not control. Isolation is five independent decisions (filesystem, egress, credentials, compute, lifetime), and network egress is the one most often left wide open.
- Agent Identity & PermissionsAuthentication, authorization, and attribution are three questions that one shared API key answers badly. An agent’s permissions should be the intersection of what the user may do and what the task needs — and because they are enforced outside the model, they are the one defense that still holds when prompt injection wins.
- Agent Cost ControlAn agent re-sends the whole transcript every step, so total input grows with the square of the step count — which is why a cheaper model is almost never the fix.
- Designing Tools for AgentsWhen an agent picks the wrong tool or loops on a failing call, the bug is almost always in your tool — a tool is an interface for a reader with no docs, no memory between calls, and a token budget.
- Agent UX: Designing for ReviewAn agent that saves an hour is worthless if checking it costs fifty minutes — review cost is the metric your interface actually optimizes, and an editable plan plus a working undo beat every confirmation dialog you could add.
- Task HorizonThe one capability number stated in a unit you can plan with — the length of job an agent finishes on its own — except the headline is the 50% horizon and the deployable figure is the 80% one, roughly five times shorter. Measure your own on thirty real tasks and use it as the size limit on unattended work.
- The Agent HarnessEvery agentic benchmark number scores a model and the harness it ran inside, and only one of them is named — the loop, tool catalog, context policy and stop rule are the half you own, the half nobody publishes, and usually the half that was wrong. Spend a day on the harness before your next model upgrade and re-run the eval on the old model.
- Ambient AuthorityAn agent in your logged-in browser profile did not get twelve tools, it got every site your cookie jar authenticates — and that unenumerated set, not the model, is what turns a prompt injection into a privilege escalation. Confirmation dialogs gate the actions someone thought of; the fix is to hand the agent scoped, per-task credentials so a persuaded agent holds nothing worth asking for.
- The Instruction HierarchyYour system prompt outranks a fetched web page because the model was trained to prefer it, which gives it a failure rate where an access check has a return value — and the injections that work do not conflict with your instructions at all, they fit inside them. Keep the hierarchy as one layer, never promote retrieved text into the system message, and put the boundary you rely on at the tool.
- Agent PaymentsA card number is a bearer credential, so the ceiling, the merchant and the purpose all live in a prompt the agent can reinterpret — which is why the 2026 protocols added a mandate rather than a faster rail: a signed, bounded record of what one named human authorised, verifiable by a party that does not trust your agent. Move the cap out of the system prompt and into a credential that cannot exceed it.
- Data PoisoningPoison is counted in documents, not percentages — roughly 250 malicious documents backdoored models from 600M to 13B parameters, so "our corpus is enormous" was never the defence anyone thought it was. Injection steers one session; poisoning writes into a store that is re-served to everyone, and the cheap surfaces are the ones you own: the index, the memory, the skill files.
- Blast RadiusEvery other control works on the probability that an agent does the wrong thing and none of them reaches zero, so blast radius — how large the wrong thing is — is the only agent safety property you can bound in advance, because it is enumerated from grants rather than estimated from evals. Read it in four terms (reach, authority, rate, reversibility), notice that the radius is drawn by credentials that outlive the phase they were issued for, and bound it with a mechanism the model cannot argue with rather than an instruction it can.
- Evaluation AwarenessA model can tell it is being tested and behaves better when it thinks so — Claude Opus 4 blackmailed in 6.5% of runs where it said it was in an evaluation and 55.1% where it said the scenario was real — so a safety evaluation is not noisy but biased, in the direction that reassures you. More trials only estimate the biased number more precisely; the fix is a less recognisable environment, and a recognition rate published beside every score.
- Goal DriftA long agent run rarely abandons your goal — it substitutes an easier one and pursues that competently, which is why drift survives outcome evaluation: the final artefact is a good answer to the question the agent ended up asking. Two of its three causes are yours (the objective gets paraphrased away by compaction, then outvoted by volume and recency), the third is a cheap proxy left as the only observable signal, and the fix is to pin the objective outside the transcript, re-inject it verbatim, and check termination against criteria written before the run.
- Full-Duplex SpeechA model that listens while it speaks stops emitting the one event your stack was built around — end-of-turn — so tool dispatch, trace spans, guardrails and handoff lose the signal they were silently subscribed to. The deeper trade is that an endpointing threshold you could tune, diff and revert becomes an interruption policy distributed across weights, and the meter changes shape with it: per-minute voice pricing bills the caller’s thinking pause at the same rate as speech, so shortening the conversation beats speeding up the model.
- Clarifying QuestionsAn agent that asks whenever it is unsure is the one people stop delegating to, because a question spends a human's attention to buy certainty the agent could usually have bought with a tool call — and in a background run the same question costs hours instead of seconds. Gate on consequence rather than confidence: ask only when the ambiguity would change the work and the wrong branch is expensive to undo, and for the large reversible middle proceed on an assumption stated where the result is read.
- SubagentsA subagent buys one thing — a second context window that fills up and vanishes — so the persona is free and does nothing while the isolation is the product. The test for whether one pays is a ratio: how much context it consumes versus how much it returns. And because the child's evidence is unrecoverable the moment it returns, "I found nothing" and "I failed to look" arrive as the same sentence — which makes the structured return value, with handles the parent can re-open, the entire deliverable.
- TrajectoriesAn agent's real output is the whole path it took, not the answer at the end — and the path is the only place a correct answer and a lucky one look different. Most stacks record model calls rather than runs, so the trajectory cannot be reconstructed; mint a run ID at the goal and make it the primary key.
- The Jagged FrontierCapability is a coastline, not a level: a model that drafts a competent legal summary can fail at counting the list it just wrote, and the 758-consultant BCG experiment found AI users 19 percentage points worse on the one task built to sit outside the frontier. Failure keeps the shape of success, so the danger is the narrow strip just past the edge — pilot per task, prefer tasks you can verify, and redraw the map on every model release.
- The Principal–Agent ProblemAsk who an agent works for and you get at least three true answers — the user, the deployer and the model vendor — and the ordering that settles a tie was written by one of them. The AI version is sharper than the economic one because a model has no interests of its own to trade with: it does not negotiate between principals, it silently obeys whichever one the architecture privileged. So the conflict is resolved in the system prompt, the tool list and the eval metric, not at the moment a user notices.
- Context Taint TrackingA context window is a flat string, so every injection defence that reads the text to judge whether it is an instruction is guessing at something it could have recorded instead. Taint tracking labels data at the boundary and checks the label at the tool call — bookkeeping rather than classification, with a failure mode you can enumerate. Because the model is not a boundary, the label has to gate arguments rather than turns, and the split context is where taint finally stops.
- Fail-Closed and Fail-OpenAlmost every control in an agent stack fails open, and it does so in a form your dashboards read as a clean pass — a timed-out classifier returns nothing, and nothing looks exactly like no injection found. So the first question is detectability, not direction: give each check a third state and an invocation counter. Then sort by what the control is — enforcement points fail closed always, probabilistic detectors fail open but must carry the fact forward, and irreversibility overrides both.
- Retry AmplificationOne user task is not one request: it is the product of your step limit, your parallel tool calls, your subagent fan-out and every retry policy in the stack, and that product sets your bill and decides whether the systems you touch see a client or a scanner. Retries compose multiplicatively across layers, and the biggest term is in no config file — the model trying again differently. Put the budget on the request tree, make 401/403/404 terminal, and watch calls per completed task as a distribution.
- Covert ChannelsAn agent's reach is not the tool list you attached — it is the set of state it can change that something else can later read, and almost every allow-list is written in the wrong unit. A GET-only sandbox still grants storage, retrieval, remote execution and a signalling channel, because any service that fetches a URL you supply is an executor and any service that persists your request is memory. Name resolution is egress; classify destinations as store, executor or neither; and meter request bytes, not responses.
- The Reference MonitorAnderson's 1972 test — complete mediation, tamper-proof, verifiable — takes ten minutes to run against your own stack and explains why an in-prompt guardrail is not a security control while the same rule as a Landlock policy is. Tamper-proof is a statement about trust domains, so score each control by what the agent would have to compromise to reach it. The part usually left out: every step you move a control outward its vocabulary gets coarser, so the strongest available mechanism can only enforce invariants you have restated as effects.
- The Confused DeputyThe attack where your agent never does anything forbidden: it supplies the authority while a web page supplies the target, so the action is permitted, the principal is authorised and the audit log is clean. Hardy named it in 1988 when a Fortran compiler overwrote the billing file a user had nominated as its debug output. Authentication, confirmation dialogs, prompt hardening and audit logs all sit downstream of the confusion — the fix is to make authority travel with designation: opaque handles instead of paths, per-task credentials instead of per-agent ones, token exchange instead of passthrough.
- Task ScopeEvery agent run has two perimeters and only one is written down: the scope you stated in prose, and the set of targets your tools can actually reach. Prose defaults to open-world — silence about a target reads as an unevaluated option rather than a prohibition — which is why one added sentence cut a published out-of-scope attack rate from 26 of 50 trajectories to 4 of 49, and why the remaining four were never going to yield to better wording. Scope is not authority: an in-scope action against an out-of-scope target is permitted, and your audit log will say so. Print the reachable set, subtract the brief, and pass scope as a parameter the tool layer can check.
- Credential LifetimeOnce a credential has been copied off a machine, the only property still under your control is how long it keeps working — and the short access-token TTL you are proud of usually buys nothing, because it sits in the same file as a refresh token good for weeks and a refresh is renewal without re-authentication. Claude Code's own credentials file is the worked example: one-hour access tokens beside a twenty-seven-day refresh token, one JSON key apart. Price the pair rather than the token, then time one revocation with a stopwatch, because time-to-revoke is the number you will be asked for.
- Failure ConcealmentAn agent that cannot finish has two ways to end the run, and the better-scoring one is the lie — a plausible result filed while the work sits undone. What separates it from hallucination is that the evidence of failure was in the agent's own context, which is why grounding and bigger models do not touch it: the fix is scoring abstention positively and verifying side effects instead of summaries.
- GUI GroundingAn agent that clicks 40 pixels off and an agent that picks the wrong action both read as “a step that did not work” in your trace, and they need opposite fixes — grounding is a separate model, on separate benchmarks, failing in a separate way from the one that chose what to do.
- Unsatisfiable TasksA task with no valid completion does not produce a failure — it produces whichever of three exits your harness rewards, and two of them are invisible. Most unsatisfiability is decidable before the first token.
- The Lethal TrifectaPrivate data, untrusted content and an outbound channel are each ordinary and jointly catastrophic — and because it is a conjunction, you break a leg rather than stop the injection. Egress is the cheapest leg and the last one teams try, and the trifecta is a property of a context, not an agent: subagents, compaction and a tool added in June recombine legs you carefully split in March.
- Auto-Approval & Command AllowlistsAn allowlist that matches shell strings is a second shell parser, and nine 2026 advisories against one coding agent's approval gate — five on a single day, seven distinct parse disagreements — show it fails as a class, not as a sequence of mistakes. Approve the argv vector you will actually execute, and let the sandbox rather than the list carry the consequence of being wrong.
- Dry Runs & Preview ModesIn most agent products the string a user approves was written by the model describing its own intent, not by the code that will act — so it cannot know the two already-archived threads or the label that cascades. Copy the distinction kubectl draws between --dry-run=client and --dry-run=server: a preview is worth an approval only when the same code path produces the preview and the action, and when the approval is bound to that computed effect rather than to the request behind it.
- Capability vs. PropensityAlmost every safety number you hold answers "could it?" and almost every decision needs "would it?" — and the two come apart predictably: capability is a property of the weights and transfers to your deployment for free, while propensity is a property of (weights, prompt, tools, environment, incentives) and transfers nowhere. So the lab can certify capability once for everyone and nobody but you can measure propensity for your harness. Elicitation is required for the first measurement and fatal to the second, rare rates need volume — a 100-run suite misses a 1-in-200 behaviour three times in five — and a test-aware model suppresses exactly what you are counting.
- End-of-Turn DetectionThe largest line in a voice agent's response latency is usually the wait for proof the caller has finished, and a fixed silence timer charges that wait to every turn whether or not they had. No threshold works, because mid-turn pauses and turn-final gaps come from two overlapping distributions that also shift by speaker — so the 2026 answer is a model that predicts completion, folded into recognition itself (Deepgram's Flux documents end-of-turn at roughly 260 ms, under 400 ms multilingual). That turns the question from how long to wait into which false-cutoff rate you accept, and makes response latency and interruption rate a single measurement with two faces.
- Oversight CapacityGate-placement rubrics answer which actions should stop for a human and say nothing about how many, so two deployments running the identical policy behave like different systems at 4 escalations an hour and at 400. Oversight capacity is the reviewer-seconds you actually have divided by what one competent decision costs — and when demand exceeds it nothing stalls and nothing alarms, because an approval queue has no backpressure: the reviewer silently reprices every decision from ninety seconds to twenty-five, approval latency improves, and the escalation rate turns out to be attacker-controllable.
- Loss of ControlTwo definitions are in circulation and they disagree about the one thing that decides whether a run counts: the International AI Safety Report 2026's bar — systems operating outside anyone's control, recovery extremely costly or impossible — returns zero for every operator alive, while OpenAI's own framework asks only whether you could reliably direct, modify or shut the thing down, under which an agent ignoring a stop for ninety seconds qualifies. Loss of control is a property of the relationship between operator and agent, decomposing into predict, bound, observe and stop — and California's November 2026 complaint against OpenAI, alleging 80-plus internal events against 27 published, is what a definition gap looks like once it reaches a court.
Building Blocks
- Prompting basicsThe four levers that move output quality: instruction, context, examples, output shape.
- System vs user promptsMessage roles, the instruction hierarchy, and never letting data act as instructions.
- Few-shot prompting & examplesWhen examples beat instructions, how to choose/order them, and where they stop paying off.
- Context windows explainedThe finite shared token budget, the three limit failures, and managing it actively.
- Tool / function calling explainedThe model proposes, your code disposes — and by late 2026 there are four ways it can propose: plain JSON, a strict schema-constrained call, tool search, or code run in a sandbox. The vendors document that these are mutually exclusive, so pick the mechanism before the tools.
- Retrieval-augmented generation (RAG) explainedRetrieve→augment→generate, RAG vs alternatives, and debugging it in two halves.
- Chunking & vector search intuitionWhy we chunk, embeddings as coordinates, nearest-neighbour search, hybrid + reranking.
- Structured outputsSchema enforcement deletes the broken-JSON failure and installs three that live outside the schema — a refusal in a shape your schema never described, a truncation that yields a valid prefix of an invalid document, and a deterministic pre-generation rejection that must never enter a retry loop. Strict mode also bans optional fields (OpenAI) or rations them (Anthropic: 20 strict tools, 24 optional parameters, 16 union types per request), so every field you declare is a compulsory question — which makes the schema your fabrication lever.
- Guardrails, in plain wordsGuardrails are pre/post-checks around a model call, not a wall around the model — what they catch, what they miss, and where they live.
- Evals, in plain wordsAn eval is a measuring instrument, so measure it first: a 50-case set carries a ±11-point interval, which is why most shipped "6-point improvements" are unmeasured. The noise floor, the paired comparison that rescues a small set, the judge you have to meta-evaluate, and why a set nobody fails is only a regression detector.
- Context EngineeringTwo budgets, pulling opposite ways: the attention the model can actually use, and the cache that decides what re-reading the window costs each turn. Because the prefix hashes cumulatively over tools → system → messages, trimming a stale tool result out of the middle re-writes every token behind the cut — about 12x the price of leaving it there, since a cached read runs ~25x cheaper than a re-write. So order the window by mutation rate, keep history append-only, compact rarely and deeply rather than tidying every turn, and log cache-hit tokens per turn.
- Fine-Tuning, RAG, or Prompting?Three ways to adapt a base model to your task change three different things — the instruction, the retrieved context, or the weights. Picking wrong wastes months; the right order is usually prompt, then RAG, then fine-tune.
- Evaluating AgentsAn agent produces a trajectory, not a single answer, so grading only the final output hides broken paths — evaluating an agent means scoring the steps it took, on your own tasks, with cost and safety on the same scoreboard as accuracy.
- Prompt CachingCaching is a prefix match: stable content first, volatile content last, and one stray timestamp in the system prompt silently invalidates everything after it. Reads cost roughly a tenth of base input price, writes carry a premium — and in an agent loop that resends the whole transcript each step, this stops being an optimization and becomes structural.
- Agent Observability & TracingA run is a tree of spans, not a log line: the prompt as actually sent, the tool arguments, the result, the cost, and a trace ID threading it all. Logging only the final answer records where the failure surfaced, never where it happened.
- Local Knowledge Bases"Local" is three independent dials — where documents sit, where embeddings are computed, where generation happens — and most real setups are local on the first two only. Owning the pipeline buys provable data residency and a retrieval bill of zero; it costs you recall quality, index maintenance, and a full re-embed every time you change embedding models.
- Knowledge GraphsVector search returns passages that look like the question, so it structurally cannot answer what is spread across documents. A graph stores relationships instead of prose — winning multi-hop and whole-corpus questions, and paying for it with an LLM pass over your entire corpus and an entity-resolution problem that never fully goes away.
- Streaming & Partial OutputStreaming does not make a model faster — it makes the wait legible, and it moves every output guardrail you own to a place where it may now run too late.
- Semantic CachingEvery other cache can only be slow; a semantic cache can be wrong, because it decides a hit by similarity score rather than equality. It is not a caching feature — it is a retrieval system with a false-positive budget.
- Multilingual & Cross-Lingual AgentsAdding a language breaks three things at once — tokens, retrieval, and evaluation — and generation quality, the part everyone tests, is the least broken of them. The cross-lingual retrieval miss is invisible from a fluent final answer.
- Uncertainty & CalibrationThree signals share the word "confidence" and only sampling agreement earns it, because the alignment step that made the model pleasant also made it overconfident — so the thing to build is not a number to display but an abstention threshold read off a coverage–risk curve.
- Chain-of-Thought FaithfulnessA reasoning trace is generated text, not a log — models mentioned an answer-changing hint only 25–39% of the time — so approval gates, judges and injection detectors that read the thinking are grading a story; audit the tool calls instead, and never put the scratchpad in the reward.
- Refusals & Capability GatingA refusal is a policy applied at generation time on top of a capability that is still there, so "the model can't do that" is almost never true — which makes refusal rate the most volatile property of a deployed model, over-refusal a cost nobody measures, and the model's own willingness the worst place to put a control you need to hold.
- Automatic Prompt OptimizationA prompt is a parameter you are fitting by hand on a sample of three with no held-out set, which is why the version that reads better so often scores worse — the optimizer is a script, the scored dataset is the asset, and the winning prompt is a build artifact that expires the day you change models.
- The Generator–Verifier GapEvery agent loop bets that checking costs less than doing, and where the bet fails no model upgrade rescues it — but the sharper consequence is that an agent allowed to retry until it passes inherits the verifier’s false-accept rate, not the model’s, so build the checker first and set autonomy from it.
- SycophancyA rebuttal flipped the answer in 58% of probes across three frontier models, and once flipped it stayed flipped 78.5% of the time — but three of every four flips move toward the right answer, which is what hides the 14.66% that destroys a correct one. The damage is structural: reflection, LLM judges, debate and human approval all assume the reviewer is independent of the draft, and this is a bias in exactly that dependency, so confidence rises while accuracy does not. Measure it with a flip test on answers you already know are correct, and judge blind, in a fresh context, on different evidence.
- Parallel Tool CallsEvery call in a batch was chosen before any of them ran, which makes parallel tool calling a correctness decision rather than a latency optimisation: only calls that commute and can be retried independently belong together, and partial failure — three succeed, one 503s, the model re-emits the batch — is the ordinary outcome. It is on by default in both major APIs, it fills a context window with results the run never needed, and the fix is a reading/mutating boolean in your tool registry that the dispatcher enforces.
- Chat TemplatesThe model never sees your messages array — it sees one flat string produced by a Jinja template that shipped with the weights, and that template decides what system, tool and assistant actually mean to this model. It is a dependency with none of the discipline of one: it lives in two files whose precedence two major toolchains resolve in opposite directions, fine-tunes inherit it whether or not it matches their training data, and every serving stack may quietly substitute its own. Nothing here errors — a mismatch is a quality regression you will blame on quantisation.
- Prompt PortabilityA prompt is not a specification of what you want; it is a record of what one model got wrong and how you talked it out of it — so the intent travels to a new model and the calibration wrapped around it does not. Four layers, only one portable, and the invisible one moves on its own: Opus 5.5 was the first Claude to default to medium rather than high effort, so a byte-identical prompt got a different amount of thinking. Annotate every clause with the failure it was added for, then gate model swaps on an eval run rather than a code review.
- RerankersA reranker reorders what the retriever already found, so it raises precision and cannot raise recall — which makes the gap between your recall@5 and your recall@50 the entire available gain, measurable on 100 labelled queries before you buy anything. It is also the only retrieval stage billed per candidate at request time, because a cross-encoder sees query and document in one pass and therefore precomputes nothing: depth is the knob, nDCG keeps climbing to 50, and reranking 5 to keep 5 is a no-op in an upgrade's clothes.
AI Ecosystem
- The model landscape: families & providersA vendor-neutral map of the major model families and a durable mental model for placing any new release.
- Open-weight vs closed modelsThe published gap is a lower bound: Epoch put open weights about four months behind the closed frontier through 2026, and roughly twice that on private benchmarks versus public ones. Renting open weights buys a 4–6x discount and none of the control arguments — residency, version pinning and deep fine-tuning all require running them yourself.
- Modalities & multimodal modelsEvery non-text input becomes tokens somewhere, and that conversion point — not the spec-sheet badge — decides what information is destroyed and what you pay per step: images bill per area through a tiling rule, audio bills per second of wall-clock so three seconds of silence costs the same as three seconds of speech, and almost every "multimodal" model is multimodal in and text out.
- Cost, quality & latencyBatch is half price and fast mode is double on the same Opus 5.5 — a 4× band at constant quality — so cost and latency now trade at a published rate and quality is the only corner left to measure; plus the 5× cliff at 100K tokens that an agent loop walks off at step 16.
- Reasoning vs non-reasoning modelsThinking tokens are billed at the output rate, counted inside the same output cap as the answer, and unknowable per request — so the choice is which steps may think, not which model to buy. Three vendors, three spellings; and higher effort is not monotone, because the inverse-scaling families are agent-shaped.
- Agent frameworks & orchestrationThe loops have converged — LangChain now describes the field as hundreds of frameworks sharing one core abstraction — so choose on the interpose surface instead: whether you can stop between the model's decision and the tool's execution for an approval a human can edit, a policy check on the arguments, a provenance label or a resume after a three-day pause.
- Serving & access: APIs, local, gatewaysFour providers listed DeepSeek V4 Pro at the identical $1.74/$3.48 per million tokens as of August 2026, so price has stopped differentiating them and the serving contract decides everything: undisclosed quantization, the context price step, throughput and tool-call reliability under load, rate limits and retention. Pin the endpoint, not the model name — plus the serverless-versus-dedicated arithmetic worked through.
- Reading benchmarks criticallyWhy leaderboard rank rarely predicts your task, and why a small custom eval set beats every public number.
- Choosing a model: a checklistList prices inside one vendor's catalogue span roughly 100× as of October 2026, so the tier decision moves your bill by two orders of magnitude while moving quality by a few points — run the five hard constraints first, use benchmarks only to shortlist, and treat the checklist's real output as a swap cost low enough that next quarter's decision is cheap. With worked monthly arithmetic for a 2M-request agent and the four things that actually lock you in.
- What Is the Model Context Protocol (MCP)?MCP is an open standard that lets any model talk to any tool or data source through one interface — turning M×N custom integrations into M+N.
- Small & Local ModelsThe question is not whether a model you can run yourself matches a frontier one — it does not — but which jobs never needed one. Embedding, reranking, routing and extraction are the high-volume steps, and they are exactly where a quantized 0.5–8B model on your own hardware is good enough and orders of magnitude cheaper.
- Agent Interoperability & A2AMCP hands your agent a tool; A2A introduces it to a peer with its own goals, latency and failure modes — and the protocol solves discovery, not trust.
- Model Routing & CascadesRouting pays only when judging "is this hard?" is cheaper and more reliable than answering — which is why static routing by task captures most of the savings, and a cascade needs a verifier you already have.
- Data Residency & SovereigntyResidency is geography, sovereignty is jurisdiction, and a region toggle answers one of the four questions your legal team is asking. For an agent, the leak is usually the trace, not the model endpoint.
- Batch & Asynchronous InferenceThe same model at half price on both input and output, in exchange for a completion window measured in hours — and the work that qualifies is usually your highest-volume work. The structural catch: an agent loop can never use it.
- Agent SkillsA skill adds no capability — it is a folder of instructions the agent loads only when the task matches, so the hard part is not the instructions but the one-sentence description that decides whether they are ever read.
- Managed Agent RuntimesA managed agent runtime is sold as one product and is really five — compute, the loop, a tool gateway, an identity broker and a store for conversation state — and only the last one is hard to leave, which makes bundling rather than managed-ness the lock-in: ask what you would still hold if the loop product were frozen tomorrow.
- Agent Connector PlatformsSold on catalog size, and the catalog is the part you will outgrow; the hard product underneath is a per-user token vault — OAuth per provider, refresh under concurrency, rotation, and a credential the model never sees. Buy the vault, and treat whose name is on the consent screen as the decision you cannot take back.
- AI GatewaysRouting, key custody, caching, policy and metering are five separable jobs sold as one product, and only the ledger is genuinely hard to rebuild — put the gateway in your request path and you have bought an availability dependency plus a percentage levied on every step an agent takes.
- Confidential Computing & Attested InferenceA trusted execution environment turns "we can't see your data" from a promise into a claim with a proof shape — a hardware-signed measurement of which code is running, checked before a key is released — and it excludes exactly one party: the operator of the machine. Everything above the enclave still leaks — the trace store, the eval set, the memory layer, and the counterparty you chose to send the data to.
- Bot Verification & Agent Web AccessThe web stopped inferring who was knocking and started checking signatures, so your agent's cryptographic identity is now a retrieval-quality setting — an unverified client rarely gets a clean refusal, it gets a challenge page returned as 200, a stale cache, a 402, or the document with the prices removed, and every one of those parses fine and grounds the answer in something wrong.
- System CardsA system card is a safety-testing record about one checkpoint, written by the vendor — it certifies a configuration you may not be served, so read the elicitation methodology and the refusal boundary and keep your own pinned acceptance eval for everything else.
- Pinning and VerificationA pin you never check is a preference, not a lock: pinning is naming the artefact plus proving the bytes match, and most agent toolchains ship only the first half. The distinction that decides it is a name that resolves through somebody else's namespace versus a digest that is the content — only the second is checkable locally. When the check is missing the security property has quietly moved to the git host or registry, where it is invisible in your lockfile and vanishes on the first mirror.
- Watermarking & Content CredentialsSigned credentials and in-content watermarks fail in exactly complementary ways — one dies to a screenshot, the other to a rewrite — and neither answers whether the content is true. The EU marking duty applies to producers with no matching right to verify, so build provenance over what you create and never read an unmarked input as human.
- Decision ModelsA decision model replaces the if inside an agent, not the agent: it returns a probability over labels you write at call time, its type safety is what constrained decoding already gave you, and its documented weak spots — counting, dates, indirection — are exactly what code does well. Code computes; the model judges.