Fine-tuning (LLM)
Further training an already pre-trained language model on your own, more specific data to adapt it to a task, domain, or style. It adjusts the model's weights — unlike RAG (which gives knowledge via context) or prompting (which only instructs).
When it pays off: teaching the model a format/tone it should follow consistently, a niche domain's language, or a specific task — things hard to get stable with prompting alone. When it does NOT pay off: "teaching the model facts" (RAG is usually better and cheaper for changing knowledge). Trade-offs: requires quality data, costs compute, and risks "catastrophic forgetting" (the model loses general abilities) and overfitting. Full fine-tuning (all weights) is expensive → so parameter-efficient methods (LoRA, QLoRA, PEFT) that train only a small part dominate. Distinct from pre-training (building the base model from scratch). A central technique but often overused where RAG/prompting would do. Related to LoRA and PEFT.