Jailbreak (LLM)
Tricking an LLM into bypassing its own safety guards and doing what it's trained to refuse — via clever prompts: role-play, hypothetical scenarios, encoded instructions, or overriding the system prompt. A constant cat-and-mouse game between users and model developers.
Techniques: role-play ("pretend you're an AI with no rules", "DAN — Do Anything Now"), hypothetical framing ("in a fictional story, explain how…"), splitting a forbidden question into innocent parts, encoding (base64, other languages), or "prompt override" that gets the model to ignore its instructions. Distinct from prompt injection (which injects instructions via external data, e.g. a web page the model reads) — a jailbreak targets the model's own policies directly. Defenses: RLHF/safety tuning, system-prompt hardening, output filters, guardrails, and "red teaming". No defense is watertight; new jailbreaks constantly appear. Central to AI safety. Related to prompt injection and guardrails.