The boundary follows a change in meaning

A semantic chunker commonly embeds sentences or groups of sentences, compares neighboring representations, and splits where similarity drops. Other implementations use an LLM or discourse model to identify boundaries.

The name describes the goal, not one standardized algorithm.

The technique addresses a failure of fixed windows: a boundary can cut a condition away from the instruction it qualifies or combine two unrelated sections merely because they fit the same size limit.

BOUNDARY DECISIONSemantic change proposes where one retrieval unit endsThe downstream query set still decides whether the split is useful.
Read neighboring sentence groupsPreserve their headings and source positions.
Compare their meaningUse the selected embedding, discourse, or model signal.
Apply a size guardrailCoherence does not justify an unbounded chunk.
same thought
Keep the groups togetherThe statements remain one candidate retrieval unit.
topic changes
Place a boundaryThe next group starts a separate candidate chunk.

A coherent chunk can still fail retrieval. Evaluate the boundary with real questions.

An API warning belongs with the step it limits

Imagine a procedure whose final sentence warns that token rotation invalidates active sessions. A fixed boundary may place the warning in the next chunk, while a semantic boundary can keep the consequence with the rotation step.

Late chunking changes when embeddings are computed, not how boundaries are chosen. Recursive chunking follows structural separators, and simple overlap repeats text across adjacent chunks.

Coherence does not excuse unbounded chunks

The contextual retrieval analysis shows how chunk context can be improved at index time. Semantic chunking does not apply when each database record is already atomic, and the common misuse is letting one broad topic produce a chunk too large for precise retrieval.

Boundary quality must be tested through retrieval

A chunk can read coherently and still be a poor retrieval unit. It may contain several answers, omit a heading that disambiguates the subject, or exceed the amount of text the downstream model can use effectively.

A representative query set should reveal whether the answer-bearing passage is retrieved and whether the returned chunk includes enough context to support an answer. Average chunk length alone cannot establish that the boundaries are useful.

Continue with these glossary entries: