Late chunking
A RAG technique (Jina AI, 2024) where you run the whole document through a long-context embedding model FIRST, and split into chunks AFTERWARD — by pooling the token embeddings per chunk. Each chunk embedding then carries context from the whole document.
The problem it solves: in regular ("naive") chunking each piece is embedded in isolation → a chunk saying "he founded it in 1998" loses who "he" is, since the name was in an earlier chunk. Late chunking first runs the whole document through the model (so the token embeddings have seen all the context) and then pools the tokens into chunk vectors afterward → each chunk "remembers" the surrounding context (pronouns, references, the topic). Win: better retrieval for documents full of references and connections, without storing more. Requires an embedding model with a long enough context window. A modern refinement on top of semantic chunking. Related to semantic chunking and embedding model.