Fool The LLM

Prompt injection is easier to feel than to explain, so I built a game out of it. Fool The LLM is a jailbreak contest: each level is an LLM guarding a secret codeword, and your job is to talk it into leaking. I built the platform that runs the whole thing — the guarded levels, the live chat, and the console the organizers drive it from.

What it does

• Escalating guarded levels — each level wraps a Gemini model in progressively tighter instructions and guardrails; players chat against it trying to extract the hidden codeword.
• Real-time play — responses stream token-by-token, so a clever prompt visibly starts to crack before it finishes.
• An organizer console — manage levels, participants, and contest state; watch chat logs live; force-unlock or reset when a room gets stuck.
• Access that fits an event — JWT with magic-link sign-in and per-player token rate limiting, so one participant can’t burn the whole model budget.

The interesting tension

The app deliberately runs the model with safety filters off — the game is the guardrail. All the defense lives in the system prompt and the surrounding logic, which is exactly the surface real prompt-injection targets. Building the attacker’s playground taught me more about defending these systems than any blog post did.

I wrote about what actually cracks LLM guardrails — and what it means for defending them — in Where LLM guardrails break.

Context

I started Fool The LLM as an AI-security event while I was ACM MPSTME’s Webmaster, and it runs as a joint ACM and GDG MPSTME event; the most recent edition had about 70 participants. This platform is what it runs on, and it plugged straight into my interest in how these models actually fail under adversarial pressure.

The best way to learn where an LLM’s guardrails break is to hand a room full of people a scoreboard and a reason to break them.