Context compression
Shrinking the retrieved material in a RAG pipeline before it's sent to the model — by filtering out irrelevant parts or summarizing. It saves tokens (cost/latency) and reduces the noise that would otherwise distract the model.
Why: a coarse retrieval often pulls in chunks that are only partly relevant, plus pure noise. Sending it all raw wastes the context window, raises the cost, and can actually worsen the answer (the model "lost in the middle" or distracted by irrelevant text). Context compression runs an intermediate step that extracts only the relevant sentences from each chunk, or summarizes them, so only the essentials reach the model. Methods: a smaller LLM that extracts/summarizes, extractive filtering per sentence, or relevance filtering with a reranker. Trade-off: an extra step (latency/cost) that pays off when retrieval is noisy. Related to reranking but focused on shortening, not reordering. Related to reranker and context window.