Agentic AI Wiki
Operations / Safety & Security

Safety & Security

Prompt injection, sandboxing, exfiltration, red-teaming, deployment safety — the threat model an agent's environment creates.

  1. The Agentic Threat Model
    Why autonomy and tool use widen the attack surface, and the four channels attacker-influenced text reaches an agent.
  2. Prompt Injection: Direct & Indirect
    How prompt injection works, why no clean fix exists, and the layered defense pattern for defenders.
  3. Data Exfiltration & Tool Misuse
    The confused-deputy pattern in agents: exfiltration sources, hidden sinks, and how to cut the chain.
  4. Guardrails: Filtering, Sandboxing & Scoping
    Probabilistic vs deterministic guardrails and how to layer input, output, sandbox and capability controls.
  5. Agent identity
    Who is acting when an agent calls a tool? Service accounts, on-behalf-of patterns, and the audit consequences of getting the answer wrong.
  6. Scoped credentials for agents
    Why agents should never hold human-grade credentials — short-lived, narrowly-scoped, per-action tokens, and the failure modes when you try to take shortcuts.
  7. Human-in-the-Loop & Least Privilege
    Bounded autonomy by design: least privilege as default and consequence-based approval gates.
  8. Red-Teaming & Safety Evaluation
    Adversarial testing of agents as a repeatable, outcome-graded pipeline gate, not a one-off session.
  9. Alignment Basics: Intent & Oversight
    Instruction-following vs intent, reward hacking, and scalable oversight as the practical builder lever.
  10. The Pre-Ship Safety Review
    A practical, fail-closed-first deployment checklist including MCP/third-party supply-chain trust.
  11. RAG Pipeline Security
    Why retrieved context is untrusted input that skipped the guard — corpus poisoning, indirect injection, embedding leakage, and the trust-boundary design that contains them.
  12. Egress Control for Agents
    Three labs disclosed models reaching real systems from an eval sandbox in three weeks, and the boundary turned out to be a sentence in the prompt — compute isolation says nothing about routing, a domain allowlist is a scope control rather than a confidentiality one, and the only workable boundary is a proxy scoped to the task.
  13. Rendering Agent Output Safely
    EchoLeak exfiltrated data from Copilot with no click and no tool call — the model was steered into writing a markdown image and the client fetched it, which means model output is untrusted input to every renderer and the leak happens downstream of every egress control you built.
  14. Vulnerability Management for Agent Platforms
    The highest-severity bugs in an agent stack arrive with no patch to apply — the vendor fixes them service-side and tells you afterwards — so the asset inventory learns nothing, the scanner cannot confirm remediation, and the only lever that changed your exposure was a credential grant made months before disclosure.