LangChain splitters vs Chonkie vs Docling vs LlamaIndex: the chunker is a schema

langchain-text-splitters still defaults to 4,000 characters counted by len() — a unit no retriever uses. Pick the chunker whose unit is your embedding model's tokenizer, then pin it, because a boundary change silently invalidates every stored vector, citation offset and eval number you have.

By Agentic AI Wiki 15 min read

The most widely used chunker in the ecosystem still defaults to 4,000 characters, measured by Python's len() — and no embedding model has ever had a character limit. That mismatch is the whole subject: pick the library whose unit is your embedding model's tokenizer, then pin it, because a chunk boundary is a schema and changing it invalidates every vector, citation offset and eval number you already stored.

At a glance

Four libraries, read at the versions on PyPI on 18 November 2026. They are not four implementations of one idea; they disagree about what a chunk is measured in, which is the only axis that changes your retrieval numbers.

LibraryVersion (18 Nov 2026)Chunk unitLicence
langchain-text-splitters1.1.3 (2 Oct 2026)Characters, via len()MIT
chonkie1.7.0 (7 Jul 2026)Tokens, or raw bytesMIT
docling-core (HybridChunker)2.107.1 (16 Nov 2026)Tokens, your tokenizerMIT
llama-index-core0.14.26 (13 Nov 2026)Sentences, scored by embeddingsMIT
Four chunking libraries compared on three axes A four by three matrix. langchain-text-splitters measures in characters, reads no document structure, and reproduces boundaries if the version is pinned. Chonkie measures in tokens or bytes, reads structure only for code and tables, and reproduces boundaries if the version is pinned. Docling HybridChunker measures in tokens from your own tokenizer, is layout-aware, and needs both the tokenizer and the version pinned. LlamaIndex SemanticSplitter measures in sentences scored by embeddings, is sentence-level only, and its boundaries depend on the embedding model. Legend: strongest on this axis, partial, weakest or absent. What decides a chunk boundary, and whether you can reproduce it Unit it measures in Reads document structure Boundary reproducible langchain-splitters Characters, via len() No Pin the version Chonkie Tokens, or raw bytes Code and tables only Pin the version Docling Hybrid Your embedder's tokens Yes, layout-aware Pin both, plus default LlamaIndex Semantic Sentence similarity Sentence level only Embedding-dependent Strongest on this axis Partial Weakest or absent
All four are MIT-licensed and actively released. They differ on the unit, and on whether you can reproduce last year's boundaries.

The default is the finding

Read langchain_text_splitters at the exact released tag and the base class says this:

class TextSplitter(BaseDocumentTransformer, ABC):
    def __init__(
        self,
        chunk_size: int = 4000,
        chunk_overlap: int = 200,
        length_function: Callable[[str], int] = len,
        ...
    ) -> None:
        """
            chunk_size: Maximum size of chunks to return
            chunk_overlap: Overlap in characters between chunks
        """

RecursiveCharacterTextSplitter — the one every tutorial reaches for — declares no size of its own. It inherits these, so out of the box it cuts at 4,000 characters with 200 characters of overlap, and the docstring says "characters" in as many words.

Why that matters is arithmetic, not taste. English prose runs roughly four characters to a token, so 4,000 characters lands near 1,000 tokens and fits comfortably inside any modern embedding model's window. Minified JSON, source code, and Chinese or Japanese text do not run at four characters to a token; they run much closer to one. The same default that produces a comfortable 1,000-token chunk for a blog post produces a chunk several times over the model's limit for a CJK corpus, and the embedding call silently truncates — which means the tail of the chunk is in your index as text and absent from the vector that retrieves it.

The library is not wrong to default to characters; len() is the only length function available without knowing your tokenizer. The error is downstream, in treating the default as a recommendation. Token counting is one argument away — TokenTextSplitter, or .from_tiktoken_encoder(), or .from_huggingface_tokenizer() — and the reason to take it is that your retriever's unit and your chunker's unit should be the same unit.

langchain-text-splitters — the widest menu, the weakest default

What it has is coverage. The package exports 20-plus splitters including MarkdownHeaderTextSplitter, three HTML splitters, RecursiveJsonSplitter, per-language code splitters, and tokenizer-backed variants. For a heterogeneous corpus where you need a different strategy per file type and do not want four dependencies, this is still the pragmatic answer.

Where the structure-aware splitters actually help

The header splitters are the under-used part. MarkdownHeaderTextSplitter cuts on heading levels and attaches the heading path as metadata, which gets you most of the benefit people chase with semantic chunking at none of the cost — a chunk that knows it came from Pricing → Enterprise tier is a chunk a reranker can use. Note that it and HTMLHeaderTextSplitter do not derive from TextSplitter, so they do not take chunk_size at all; they are a first pass you then feed to a size-bounded splitter.

SemanticChunker is not in this package

Worth saying plainly because it moves people's dependency graph: LangChain's SemanticChunker lives in langchain-experimental (0.4.2, released 22 May 2026), not in langchain-text-splitters. Its own module docstring calls it "Experimental text splitter based on semantic similarity", and it credits Greg Kamradt's notebook as the source of the method. Its defaults are buffer_size=1 and a 95th-percentile breakpoint threshold.

Chonkie — token-native, and the only one with a byte-level escape hatch

Chonkie 1.7.0 ships twelve chunkers and, unlike LangChain, its units are tokens by default. The list at this version is TokenChunker, FastChunker, SentenceChunker, RecursiveChunker, SemanticChunker, LateChunker, CodeChunker, NeuralChunker, SlumberChunker, TableChunker and TeraflopAIChunker. Two of those are interesting beyond their names: LateChunker implements the embed-then-split ordering described below, and SlumberChunker — which the README also calls "AgenticChunker" — puts an LLM in the chunking loop.

Read the performance claims as vendor numbers

The README leads with "33x faster than the slowest alternative" for token chunking and a 505 KB wheel against "1-12MB for alternatives". The repository's own BENCHMARKS.md is more useful and less tidy: on 100,000 Wikipedia articles it reports Chonkie at 58 seconds for token chunking against LangChain's 1 min 10 sec — a 1.21× gap, not 33× — and LlamaIndex at 50 minutes, which is where the large multiple comes from. The install-size figures also disagree between the two files (49 MB in the README, 15 MiB default in BENCHMARKS.md). These are self-reported marketing numbers measured by the vendor; the 1.21× against LangChain is the one to plan with, and for most corpora chunking is not the bottleneck anyway.

FastChunker is the genuinely novel entry

A SIMD-accelerated byte-based chunker, included in the default install, is an odd thing to want until you are re-chunking a corpus large enough that tokenizer overhead dominates. It is the right tool for a first-pass bulk split and the wrong one for anything you will embed directly, because bytes are further from your retriever's unit than characters are.

Docling HybridChunker — the only one built around your tokenizer

Docling inverts the design. Its HierarchicalChunker emits one chunk per detected document element using the parsed layout, and HybridChunker then applies, in the docs' words, "tokenization-aware refinements on top of document-based hierarchical chunking" — one pass that splits chunks "only when needed (i.e. oversized w.r.t. tokens)", and a second that merges "undersized successive chunks with same headings & captions", controlled by merge_peers (default True). It also repeats table headers into each chunk when a table spans several, via repeat_table_header (default True).

The default tokenizer is the trap

Docling's example notebook states the rule: "In a RAG / retrieval context, it is important to make sure that the chunker and embedding model are using the same tokenizer." But the tokenizer argument is optional, and the default is sentence-transformers/all-MiniLM-L6-v2:

class HybridChunker(BaseChunker):
    tokenizer: BaseTokenizer = Field(default_factory=_get_default_tokenizer)
    repeat_table_header: bool = True
    merge_peers: bool = True

So a pipeline that embeds with a Voyage, Cohere or OpenAI model and leaves the tokenizer unset is budgeting chunks against a 384-dimension MiniLM's tokenizer and a 256-token sequence length. Nothing errors. You get chunks that are the wrong size for the model that will embed them, in the one library that was built to get this right.

The warning you will see is a false alarm

Docling's FAQ pre-empts the confusion: HybridChunker triggers "Token indices sequence length is longer than the specified maximum sequence length for this model", and the FAQ calls this "a known & anticipated 'false alarm'" — the chunker invokes the tokenizer on a long sequence precisely in order to count it, so transformers warns before the split happens. The number to check is the one the FAQ tells you to print: the longest chunk actually yielded, against the model's max length. Do that check in CI and the warning stops mattering.

LlamaIndex node parsers — sentences, and a threshold you must tune

SemanticSplitterNodeParser groups sentences and cuts where embedding dissimilarity crosses a percentile: buffer_size=1, breakpoint_percentile_threshold=95. The docstring is explicit about the direction of the dial — "the smaller this number is, the more nodes will be generated" — and the docs list two caveats rather than calling it experimental: "The regex primarily works for English sentences" and "You may have to tune the breakpoint percentile threshold."

The quiet consequence is the one that belongs in this article's argument: a semantic splitter's boundaries are a function of an embedding model. Change the embedding model and you have changed the chunk boundaries, not just the vectors — so the migration that was supposed to be "re-embed the same chunks" becomes "re-chunk and re-embed", and every citation offset you stored against the old nodes is wrong.

SentenceWindowNodeParser is the better-behaved cousin

It splits into individual sentences and stores the surrounding window in metadata (window_size=3 by default, meaning three sentences each side), invisible to the embedding model and the LLM until a MetadataReplacementNodePostProcessor swaps the sentence for its context before generation. That is a deterministic boundary with a context payload — closer in spirit to late chunking than to semantic splitting, and far easier to reproduce.

Buying back the context you cut

Chunk-first embedding compared with late chunking Two pipelines. The first splits the document into chunks and then embeds each chunk on its own, so a chunk's vector never saw the rest of the document. The second embeds the whole text first to get one vector per token, then mean-pools those token vectors over the same chunk boundaries, so each chunk vector carries context from the entire document. Chunk first — the default in all four libraries Document Split into chunks Embed each chunk Chunk vectors A sentence whose subject sits three paragraphs back loses it at the cut. Late chunking — same boundaries, context kept Document Embed whole text Token vectors Mean-pool per chunk The cut is unchanged; what changes is what each vector was allowed to see.
Late chunking leaves the boundary alone and changes only what each chunk's vector was allowed to see.

Every chunker above has the same structural problem: a chunk's vector never saw the rest of the document, so a sentence whose subject appeared three paragraphs earlier is embedded without it. Two published techniques attack this from opposite ends, and neither requires you to move the boundary.

Late chunking changes the ordering

Günther, Mohr, Williams, Wang and Xiao's Late Chunking: Contextual Chunk Embeddings Using Long-Context Embedding Models (arXiv 2409.04701, 2024) runs the transformer over the whole text first, then mean-pools the token vectors over each chunk's span rather than embedding each chunk alone. The gains in the authors' own repository are real but modest and uneven: on BeIR with 256-token chunks and jina-embeddings-v2-small-en, nDCG@10 moves from 64.20% to 66.10% on SciFact and from 23.46% to 29.98% on NFCorpus, but is identical on Quora (87.19% either way), and on NFCorpus and TRECCOVID not chunking at all beats both. The repo's own observation is the useful one: the improvement correlates with average document length. Short documents have no distant context to lose.

Contextual retrieval changes the text

Anthropic's September 2024 method prepends a generated, document-aware sentence to each chunk before embedding it. The published ladder is the most quotable set of numbers in this field, and all four rows use the same chunk boundaries:

Top-20-chunk retrieval failure rate across four configurations A horizontal bar chart of Anthropic's published figures from September 2024. Baseline embeddings fail 5.7 percent of the time. Adding contextual embeddings drops it to 3.7 percent. Adding contextual BM25 drops it to 2.9 percent. Adding reranking drops it to 1.9 percent. The chunk boundaries are identical in all four rows. Top-20-chunk retrieval failure rate (lower is better) Baseline embeddings 5.7% + contextual embeddings 3.7% + contextual BM25 2.9% + reranking 1.9% 0% 2% 4% 6%
Anthropic, September 2024: the failure rate falls by 67% without any change to where the document was cut.

Contextual embeddings alone cut the top-20-chunk retrieval failure rate by 35% (5.7% → 3.7%); adding contextual BM25 reaches 49% (5.7% → 2.9%); adding reranking reaches 67% (5.7% → 1.9%). The one-time generation cost is stated as "$1.02 per million document tokens". Read against the late-chunking table, the comparison is unkind to chunker-shopping: moving the boundary bought single-digit nDCG points, while adding context and a reranker to the same boundaries cut two thirds of the failures.

Why the boundary is a schema

Here is the argument the four libraries' docs do not make for you. A chunk boundary is not a preprocessing choice, it is a key. Three artefacts are defined against it, and all three break silently when it moves:

  • Every stored vector. The embedding in your index is the embedding of a specific span. Re-chunk and the vectors describe text that no longer exists in that shape, so the index is stale in a way no freshness check detects — see index freshness and invalidation.
  • Every citation offset. If you show users "page 4, characters 1,200–2,000", those numbers are relative to a boundary scheme. A library upgrade that changes a separator list changes where the cut falls, and your citations now point a paragraph off.
  • Every eval number. A retrieval score is a score for a (chunker, embedder, retriever) triple. Comparing this month's recall against last month's, across a chunker version bump, compares nothing — the subject of evaluating RAG.

Which makes the operational rule short. Pin the chunking library to an exact version, pin the tokenizer by name, and record both in the index metadata next to the embedding model — then treat a change to any of the three as an index migration with a backfill, not as a dependency upgrade. That is the same discipline as reindexing and embedding migrations, and the chunker belongs inside its scope.

It is also why evaluation tooling for this exists and is worth knowing about: Smith and Troynikov's Evaluating Chunking Strategies for Retrieval (Chroma, July 2024) ships a companion package that scores chunkers by token-level intersection-over-union and recall against a generated dataset, and implements ClusterSemanticChunker and LLMSemanticChunker alongside fixed-token and recursive baselines. IoU is the right metric for the question "did the cut land where the answer is", and recall alone is not — a chunker can score well on recall by returning large chunks that happen to contain the answer along with everything else.

When to pick which

SituationPickWhy
PDFs, DOCX, tables, anything with layoutDocling HybridChunkerOnly one that reads parsed structure and budgets in your tokenizer's tokens — but set the tokenizer explicitly.
Mixed corpus, many file types, one dependencylangchain-text-splittersWidest menu, including header and code splitters. Override the character default on day one.
Large corpus, re-chunked often, or you want late chunking in-libraryChonkieToken-native defaults, LateChunker, and a byte-level fast path for bulk passes.
Already on LlamaIndex, want context without changing boundariesSentenceWindowNodeParserDeterministic sentence boundaries plus a metadata context window; reproducible in a way semantic splitting is not.
Retrieval quality is the actual problemNone of the above firstAdd generated context and a reranker to your existing boundaries. The published gain is far larger than any boundary change.

FAQ

What chunk size should I use?

Ask in tokens, not characters, and answer against your embedding model's sequence length and your reranker's budget. Anthropic's own RAG primer describes chunks of "usually no more than a few hundred tokens", and their published results used 800-token chunks; the specific number matters less than that you measured it in the unit your retriever uses.

Does semantic chunking beat fixed-size chunking?

Not reliably enough to lead with. The published evidence for boundary-moving techniques shows single-digit improvements that vary by corpus — late chunking helped on SciFact and NFCorpus, tied on Quora, and lost to not chunking at all on two datasets. Both LangChain and LlamaIndex hedge their semantic splitters in their own docs, one as "experimental" and the other with caveats about English-only regex and threshold tuning.

Is late chunking the same as contextual retrieval?

No. Late chunking changes the order of operations — embed the whole text, then pool per chunk — and adds no tokens. Contextual retrieval changes the text, prepending a generated situating sentence to each chunk before embedding, at a stated one-time cost of $1.02 per million document tokens. They are compatible, and both leave the boundary where it was.

Do I need a tokenizer argument for Docling's HybridChunker?

You should pass one, even though it is optional. The default is sentence-transformers/all-MiniLM-L6-v2, so leaving it unset while embedding with a different model silently sizes your chunks against the wrong tokenizer — the exact failure the chunker was designed to prevent.

Is Chonkie really 33 times faster than LangChain?

No. That README figure compares against the slowest alternative in the set, which is LlamaIndex. Chonkie's own benchmark file puts token chunking at 58 seconds against LangChain's 1 minute 10 seconds on 100,000 Wikipedia articles — about 1.2×. All of these are vendor-measured.

Further reading

On this wiki:

Project sources: