Transformer architecture
Neural architecture that replaced RNNs and CNNs for sequence modeling. Vaswani et al., "Attention Is All You Need", Google, 2017.
Built on self-attention. Two variants: encoder-only (BERT — understanding), decoder-only (GPT — generation), encoder-decoder (T5, BART — translation). Scaling to hundreds of billions of parameters gave the LLM revolution. Modern evolutions: Mixture of Experts, Mamba (state space alternative), DiT (Diffusion Transformer for image/video). The paper is one of the most cited in computer science ever.