LLM-eval
Process att utvärdera LLM-output quality. Two-categories: benchmark-evals (MMLU, GPQA, SWE-bench — fixed datasets) + production-evals (user-traffic-specific, often LLM-as-judge). Standard ML-engineering-discipline 2024+.
Tools: OpenAI Evals, LangSmith, Braintrust, Helicone, Phoenix (Arize). Categories: factual-correctness, hallucination-rate, format-compliance, safety, latency, cost. Test-set-design: representative-of-production, hand-labeled-ground-truth eller LLM-as-judge för scale. Modern best practice: eval-driven-development — write evals first, iterate prompt + model + tools. Pre-LLM-era: ML-team used precision/recall/F1, LLM-era requires open-ended-output-evaluation = LLM-as-judge or human-rating. Skiljs från observability (latency/error-rate): eval = quality-of-output.