What does it mean to jailbreak a model?
Jailbreaking a model means tricking it into producing outputs it was trained to refuse — harmful instructions, hate speech, or dangerous information. The term borrows from smartphone jailbreaking, but the AI version works entirely through language: no code exploits required.
AI models are built with safety guardrails, typically through Reinforcement Learning from Human Feedback (RLHF), which teaches the model to decline harmful requests. Jailbreaks work around those guardrails using crafty prompts — wrapping a harmful request in a roleplay scenario, framing it as fiction, or using oddly formatted text that confuses the model's filters.
It's an active cat-and-mouse problem: AI labs continuously patch known jailbreaks, but new ones keep emerging. This is why safety researchers treat language itself as an attack surface — and why no model is considered fully jailbreak-proof.