Jina AI moved the split after contextual encoding
Florian Günther and coauthors described late chunking in 2024 to preserve document-wide context in chunk embeddings. In an early-chunking pipeline, splitting happens before the transformer sees the text.
Late chunking lets the transformer contextualize tokens across the longer document first, then pools token ranges that correspond to chunk boundaries.
A pronoun reveals what early chunking loses
Consider a section that begins, “It retries the request with exponential backoff.” If the product component named in the preceding section is absent from the chunk, an isolated embedding may represent retries but not what performs them.
Encoding both sections together can carry that antecedent into the later chunk’s token representations.
Semantic chunking chooses boundaries according to topical change. Late chunking changes when embeddings are computed.
The two techniques can coexist because they solve different problems.
Long-document support is a condition, not a footnote
The contextual retrieval analysis covers a different method that prepends generated context before indexing each chunk. Late chunking is unsuitable when documents exceed the embedding model’s usable context or when each record is already self-contained.
The common misuse is calling any post-embedding text split late chunking, even though the chunk vectors were never pooled from contextualized token representations.
Document boundaries still need a policy
Encoding an entire file is not always the right unit. A long collection of unrelated tickets may fit the model window but gain nothing from sharing context, while one chapter split across files may need a larger logical boundary.
The pipeline must also map each pooled vector back to its exact source span. Without that mapping, retrieval may find a context-aware vector but the application cannot quote or cite the text that the vector represents.
Continue with these glossary entries: