LLM distillation
Small "student" model learns from large "teacher" model — either via output logits (classic Hinton distillation 2015) or by generating training data with the teacher.
Why: 7B student can perform like 70B teacher on specific tasks ⇒ much cheaper inference. Examples: Gemma 2B (distilled from Gemini), Llama 3.2 1B/3B (distilled from larger Llama), DistilBERT (2019). Differs from quantisation: distillation changes the model, quantisation changes the precision.