Attention mechanism
Neural mechanism for focusing on relevant parts of the input. Bahdanau et al. (2014) introduced it for translation. Vaswani et al. (2017) generalized it to self-attention → Transformer.
Mathematical core: softmax(QK^T / √d) · V. Q (query), K (key), V (value) are linear projections of the input. Multi-head attention runs several attention heads in parallel. Quadratic complexity in sequence length is the classic limitation → much research on efficient attention (Flash Attention, sparse attention, linear attention, Mamba/SSMs as alternatives). Cross-attention connects two sequences (encoder-decoder); self-attention connects the same sequence with itself.