Agent Memory: Short-Term vs Long-Term

Two memories: the window you have now, and the store that outlives it.

Short-term memory — also called working or in-context memory — lives inside the context window. It is the current conversation plus any scratchpad the agent writes as it works: everything the model can literally see on this turn. Long-term memory lives outside the model, in an external store — a database, a file, a vector index — and is selectively retrieved back into the window only when it is relevant to the step at hand.

That split exists because the window alone cannot serve as durable memory. It is (1) bounded in size, so it cannot hold everything the agent has ever learned; (2) wiped between sessions, so tomorrow's conversation starts blank; (3) expensive per token, since every token in the window is re-processed on every turn; and (4) prone to degrading when overfilled — the effect variously called context rot or lost in the middle, where material buried in a long payload gets effectively ignored. Persistent memory exists to decouple what the agent knows from what currently fits in the prompt.

The four kinds of thing an agent remembers.

Borrowing a taxonomy from cognitive science, it helps to name the distinct types of memory rather than treat "memory" as one bucket. Each answers a different question:

• Working memory — the active task context, right now. The sub-goal the agent is pursuing, the file it just opened, the value it computed two steps ago. This is the short-term memory from STEP 1.
• Episodic memory — records of specific past events and interactions: what happened. "Last Tuesday the user asked me to book a flight to Berlin and I did." It is time-stamped and particular.
• Semantic memory — facts, preferences, and general knowledge: what is true. "The user prefers aisle seats." "This company's finance system runs on SQL." Timeless and reusable, not tied to one event.
• Procedural memory — how to do things: skills and routines, often the agent's own evolving instructions. The checklist it follows to file an expense report, refined each time it gets one wrong. This is memory the agent uses to improve its own behavior.

Real systems blur these edges, but the labels are a working vocabulary for asking "which kind of remembering does this agent actually need?" The memory-types deep-dive maps each type to concrete storage choices.

The loop: write, retrieve, consolidate — and forget on purpose.

A memory system is not a passive archive; it runs three core operations continuously:

• Write / encode. Decide what is worth storing. You cannot and should not store everything — indiscriminate saving just recreates context rot in a bigger container. Good memory is selective about what gets committed.
• Retrieve. Fetch the relevant memories back into the window at the right moment — not all of them, only the few that bear on the current step. This is the read path, and it usually looks like search.
• Consolidate / update. Over time, merge duplicates, summarize long histories, resolve contradictions when a new fact overrides an old one, and drop what is stale. Forgetting is a feature, not a bug: a memory that never prunes eventually retrieves outdated and conflicting junk.

Those operations are implemented with a small set of storage patterns, usually combined:

• Vector stores — turn each memory into an embedding and retrieve by similarity search. This is how an agent recalls the semantically relevant note without an exact keyword match.
• Running summaries / compaction — collapse a long conversation or tool-call log into a compact recap and drop the raw transcript, keeping the gist within budget (see context-compaction).
• Structured stores — key-value tables, SQL, or knowledge graphs for facts and the relationships between them, when you need exact lookups and clean updates rather than fuzzy recall.

Memory is not RAG — and how it lands in agent design.

Because both fetch text into the window, beginners often collapse memory and RAG into one idea. The distinction is worth getting exactly right. RAG retrieves from a mostly static, external corpus — a set of documents you loaded — to help answer a question. Memory is read-write and agent-authored: the agent writes new memories from its own experience and updates them over time. RAG is a retrieval technique; memory is persistent, evolving state that the agent itself keeps building. The overlap — and the reason they get confused — is that a memory system's read path often uses RAG-style retrieval; that shared mechanism is not sameness of purpose. The retrieval-augmented-memory deep-dive works through exactly where the two meet.

In practice you rarely build all of this from scratch. A handful of real systems package the loop: Mem0 offers a managed memory API with user, session, and agent scopes; Letta (formerly the 2023 MemGPT research project) introduced an OS-inspired tiered design with core, recall, and archival memory; Zep — whose open engine Graphiti is a temporal knowledge graph that stamps facts with validity windows — targets facts that change over time; LangMem is LangChain's memory SDK; and Cognee is a graph-based, local-first option. They differ in exactly the choices above: which memory types they model, and how they handle the write and update paths.

The takeaway for building agents: treat memory as a deliberate architectural layer, decided per system, not a switch you flip by making the window bigger. Choose what to write, what to retrieve, and what to forget — then measure whether those choices help on tasks that look like yours. When you are ready to go deeper, the wiki's Memory & Context deep-dive group carries these ideas into production detail — from write-path architectures to evaluating memory — and pairs naturally with context engineering, the discipline of curating what the window holds on each step.