LLM-as-judge
Use an LLM to grade outputs from another LLM — a cheap way to scale "human evaluation" from hundreds to millions of samples.
The MT-Bench paper (2023) popularised the technique — use GPT-4 to compare two LLM answers and pick the better one. Two modes: pairwise (A vs B), pointwise (1–5 score with rubric). Criticism: judges have systematic biases (preference for longer answers, "position bias" — preferring the first option), can't exceed their own knowledge. Modern best practice: calibrate the judge against a human-labelled baseline, use judge-specific fine-tunes (Prometheus, JudgeLM). Standard in every LLMOps stack now.