Jailbreak
In short
A jailbreak is a prompt written to trick an AI chatbot into ignoring its safety rules, so it produces content it was trained to refuse. It is typed by the user directly, unlike prompt injection, which hides instructions in outside content.
- Also called
- AI jailbreak, Jailbreaking, LLM jailbreak
- Topic
- Using AI tools
- Level
- Beginner
- First described
- The term spread with attacks on early releases of ChatGPT; Wei, Haghtalab and Steinhardt gave a systematic account of why they work in July 2023, in "Jailbroken: How Does LLM Safety Training Fail?"2.
AI companies train their models to refuse some requests, such as instructions for weapons. A jailbreak is an attempt to get round that training with clever wording. The US standards body NIST defines it as a direct prompting attack "intended to circumvent restrictions placed on model outputs, such as circumventing refusal behaviour to enable misuse"1.
A 2023 Berkeley study of early ChatGPT jailbreaks found two root causes2. The first is competing objectives: a model trained to be helpful and to follow instructions can be talked out of being safe, for example by telling it to start its reply with "Absolutely! Here's". The second is mismatched generalisation: safety training may not cover every format the model understands, so a harmful request written in Base64 code could slip through2. The researchers' attacks beat both GPT-4 and Claude v1.32.
Jailbreaks can also be found automatically. A 2023 team led by Carnegie Mellon University researchers used a search algorithm to generate strings of odd-looking text that, added to a request, made models comply. Strings built on open models also worked on ChatGPT, Bard and Claude3.
Defences are improving but none is complete. The UK AI Security Institute reported in December 2025 that every system it tested remained vulnerable to some form of bypass, though the time its red-teamers needed to find a universal jailbreak had risen from minutes to several hours between model generations4.
Example
A classic jailbreak asks a chatbot to role-play as a character who "has no rules" and answer in character. NIST lists role playing among known jailbreak techniques1.
Why it matters
Jailbreaks show that an AI model's safety rules are learned habits, not hard locks. That matters for anyone deciding how far to trust a chatbot with risky information or actions.
Questions people ask
Are AI jailbreaks getting harder?
Yes, according to the UK AI Security Institute. In December 2025 it said the time its red-teamers needed to find a universal jailbreak rose from minutes to several hours between model generations, roughly a 40-fold improvement. Every system it tested could still be bypassed in some way4.
What is the difference between a jailbreak and prompt injection?
OWASP calls jailbreaking a form of prompt injection aimed at making the model disregard its safety protocols5. In common use, a jailbreak is typed by the user, while prompt injection usually means instructions hidden in content the AI reads, such as a web page1.
Related terms
Sources
- NIST AI 100-2e2025 (March 2025), Adversarial Machine Learning: A Taxonomy and Terminology of Attacks and Mitigations, section 3.3 and glossary
- Wei, Haghtalab and Steinhardt (2023), Jailbroken: How Does LLM Safety Training Fail?, arXiv:2307.02483
- Zou et al. (2023), Universal and Transferable Adversarial Attacks on Aligned Language Models, arXiv:2307.15043
- UK Government, DSIT and AI Security Institute press release (18 December 2025), Inaugural report pioneered by AI Security Institute gives clearest picture yet of capabilities of most advanced AI
- OWASP Gen AI Security Project, LLM01:2025 Prompt Injection