04 / How It Goes Wrong
Jailbreak
A deliberate attempt by a user to talk an AI out of its rules, for example through role-play or elaborate framing, to get something it should refuse.
How you’ll hear it
“A customer got it to write a fake refund policy.”
“That was a jailbreak. The guardrail was only in the instructions.”
Avoid
Relying on instructions alone to stop determined misuse.
The longer version
A jailbreak comes from the person typing; prompt injection comes from content the AI reads. Both exploit the same weakness: instructions can be argued with. The lasting defence is limiting what the system can do and checking what it produces, not writing ever longer rules.