Perplexity (metric)
Measure of how "confused" a language model is on test data. Lower = better. Calculated as 2^cross-entropy. Standard intrinsic metric for LLM evaluation (before LLM-as-judge + benchmark suites took over).
Intuition: perplexity 50 = model is uncertain between 50 equivalent next tokens. GPT-2 small ~30 on WikiText; Llama-3-70B ~3 on the same. Lower perplexity correlates with higher quality on generation tasks. Limitation: tokenizer-dependent (models with different vocabularies not directly comparable), only on text-distribution modeling (says nothing about reasoning, factuality, helpfulness). Modern eval: HumanEval, MMLU, GPQA, alignment benchmarks give more actionable scores. Confusion: not the same as Perplexity AI (search startup).