Chunking (RAG)
Splitting documents into smaller pieces ("chunks") before they're embedded and indexed for RAG. The chunk size is one of the most decisive — and tricky — design choices: too-large pieces dilute relevance, too-small ones lose context.
Trade-off: a large chunk gives more context but a blurrier embedding (many topics in one vector → worse matching); a small chunk gives sharp matching but may lack the surrounding context needed to answer. Common strategies: fixed size with overlap (simple), by structure (paragraph/heading/Markdown section), or semantic chunking (split where the topic shifts). Overlap between chunks ensures a sentence on a boundary isn't lost. Advanced: sentence-window and parent-document retrieval fetch a small chunk but give the model a larger window around it. Bad chunking is the most common cause of poor RAG quality. Related to semantic chunking and RAG.