Knowledge Graphs — Concepts

B17

Concepts · Core Building Blocks

Knowledge graphs.

Vector search returns passages that look like the question, which means it structurally cannot answer questions whose answer is spread across documents — "which services does the team that owns billing depend on, and which of those changed last quarter?" is stated nowhere and implied everywhere. A knowledge graph stores the relationships instead of the prose, buying multi-hop and whole-corpus questions at the price of an extraction bill and an entity-resolution problem that never fully goes away.

STEP 1

Entities, relations, and why that shape matters.

A knowledge graph is a set of nodes (entities: people, services, documents, companies, symptoms) joined by typed edges (relations: owns, depends_on, authored, supersedes), with attributes on both. The atomic unit is usually written as a triple — (billing-service, owned_by, platform-team).

Compare the three storage shapes a knowledge base can take:

A table answers "what are the properties of this row?" — fast, exact, but the relationships live in join keys you have to know about in advance.
A vector index answers "what text is similar to this text?" — flexible about wording, blind to structure, and fundamentally one-hop: it returns passages, and a passage that does not contain the answer cannot be traversed to one that does.
A graph answers "what is connected to this, and how?" — traversal is the primitive, so two, three, and four-hop questions cost a query rather than a miracle.

The graph's real superpower is not lookup speed. It is that the answer to a global question can be computed rather than retrieved: "what are the recurring themes in 4,000 support tickets" has no passage to find, but it does have a community structure to summarize.

STEP 2

How graphs actually get built now.

Hand-curated ontologies are why knowledge graphs had a reputation for being expensive. The current approach is to let a model do the extraction:

Extract. Pass each chunk to an LLM with instructions to emit entities and relations, usually as constrained structured output. This is the step that costs money — you are running a model over your entire corpus, not just embedding it.
Resolve. Decide that "Platform Team", "platform-team", and "the platform folks" are one node. Entity resolution is the hard part and the main source of quality loss: under-merge and the graph fragments into synonyms; over-merge and you fuse two real people into one.
Structure. Either impose a schema up front (a small ontology: which entity types and relation types are allowed) or let the model invent types freely. A schema costs design time and buys precision and queryability; free-form is faster to stand up and drifts.
Enrich. Optionally precompute community clusters and summaries, so whole-corpus questions can be answered from summaries rather than by traversing everything at query time.

Extraction quality bounds everything downstream. A graph built from a shaky parse of your PDFs is a confident, well-structured representation of the wrong facts — and unlike a bad vector index, it looks authoritative. Sample the triples and read them before you trust queries over them.

STEP 3

What it wins, and what it costs.

Wins: multi-hop questions. Anything requiring you to follow a chain — org structure, dependency graphs, citation chains, supply chains, medical or legal reasoning over related entities.
Wins: global and thematic questions. "Summarize the main risks across this corpus" is a question about the whole corpus, and top-k retrieval answers it with an arbitrary k documents.
Wins: auditability. An edge can carry its source document, so an answer decomposes into a path you can show a human. This is a genuine advantage over "here are five passages that were nearby in vector space."
Wins: time. A temporally aware graph can record that a fact was true between two dates, which lets an agent answer "who owned this in March?" instead of overwriting history with the latest value.
Costs: build price and latency. One LLM pass per chunk at ingest, versus one cheap embedding pass. On a large corpus this is the difference between an afternoon and a budget line.
Costs: staleness and churn. Updating a graph is harder than updating an index — a changed document may invalidate edges elsewhere, and re-extraction is not idempotent because the model is not deterministic.
Costs: it is not a drop-in replacement. For "find me the paragraph that says X," a graph is a worse and slower vector index. Nearly every serious system runs both and routes.

STEP 4

Where you will meet this in an agent stack.

Graphs show up in two distinct roles, and confusing them causes most of the tooling confusion:

As a knowledge base over documents — Microsoft's GraphRAG popularized the extract-then-summarize-communities pattern; LightRAG and Cognee are open-source takes that run locally, the latter building a graph plus vector store from mixed inputs.
As agent memory — a graph of facts about the user and the work, accumulated across sessions, with temporal validity so superseded facts are retired rather than duplicated. Zep's Graphiti and the reference memory MCP server both take this shape.

For a local build, the practical note is that a graph layer roughly doubles the moving parts: you now maintain an extraction prompt, a graph store, and a vector index, and you need a router that decides which one a given question should hit. Start with hybrid vector-plus-keyword retrieval, measure the questions it fails, and add the graph only if the failures are relational — because that is the failure class it actually fixes.

The GraphRAG and multi-hop retrieval deep-dive covers the algorithms and the cost/staleness heuristic in detail; local knowledge bases covers the surrounding pipeline.