Grouped-query attention (GQA)
An attention variant where several query heads share the same set of key/value heads, instead of each having its own. It shrinks the KV cache substantially (faster inference, less memory) with minimal quality loss. Standard in Llama 2/3, Mistral, and others.
Background: in regular multi-head attention (MHA) each head has its own Q, K, and V projections → the KV cache becomes large. The extreme opposite, multi-query attention (MQA), lets all query heads share ONE single KV head → minimal cache but some quality loss. GQA is the middle path: divide the query heads into groups, where each group shares a KV head (e.g. 32 query heads, 8 KV heads). Win: nearly MHA's quality with close to MQA's memory savings → smaller KV cache, higher throughput, longer context. Since the KV cache is often the inference bottleneck, this is one of the most impactful architecture choices in modern models. Related to KV cache and multi-head latent attention.