Guardrails (AI)
Protective layers around an LLM that check and constrain what goes in and comes out — block harmful queries, filter inappropriate answers, keep the model on a topic, or enforce a format. A separate safety layer beyond the model's own training.
Why: you can't trust the model's built-in alignment to always hold (jailbreaks, edge cases), so you add external controls. Types: input guardrails (detect prompt injection, forbidden topics, PII before it reaches the model), output guardrails (filter hateful/dangerous content, check that the answer stays on allowed topics, validate format/schema), and dialogue policies (control what the bot may talk about). Implemented with rules, classifiers, or a separate "judge" LLM. Tools: NVIDIA NeMo Guardrails, Guardrails AI, Llama Guard. Trade-off: too-tight guardrails give false refusals and frustrating "I can't help with that"; too-loose ones let problems through. Standard in production LLM systems. Related to jailbreak and structured output.