You cannot jailbreak math

This used to be an argument. Now it is a literature.

Two years ago this position got you a raised eyebrow in a procurement meeting. It has since been tested by independent research groups, repeatedly, and the results are not ambiguous.

Researchers at Lancaster University and Mindgard published an empirical analysis of evasion attacks against six production protection systems, including Microsoft Azure Prompt Shield and Meta Prompt Guard. Using character injection and adversarial machine learning techniques, they reported evasion success of up to 100% in some instances, while preserving the effectiveness of the underlying attack. Not degrading it. Preserving it. The attack still worked, and the guardrail no longer saw it. The paper appeared at LLM Sec 2025 and is public as arXiv:2504.11168.

A February 2026 study from TU Munich and Mila went at the problem from a different angle: not whether judges can be evaded, but whether they can be trusted to measure anything at all. Across 6,642 human-verified labels, they found LLM-judge performance degrading to near random chance under red-teaming distribution shift. Worse for anyone relying on published benchmark numbers, they found that many reported attack successes in the literature exploit inadequacies in the judge rather than producing genuinely harmful output. The judge is not just unreliable at catching attacks. It is unreliable at telling you how unreliable it is. That is arXiv:2603.06594.

A systematic review published in January 2026 covered 78 studies and 42 attack techniques against agentic coding assistants including Claude Code, GitHub Copilot and Cursor. Its finding: attack success rates exceed 85% when adaptive attack strategies are employed, and most defense mechanisms achieve less than 50% mitigation against those adaptive attacks. The authors conclude that prompt injection requires architectural solutions rather than filtering-based ones. That is arXiv:2601.17548.

And a fourth, the Robust Judge framework from Li and colleagues, tested 15 attack methods, 7 defense strategies and 12 models. It found judge robustness varying by up to 40% based on nothing more than the choice of prompt template, and identified live vulnerabilities in a production deployment. arXiv:2506.09443.

The published evidence on using a model to guard a model

Peer-reviewed and preprint findings on the reliability of LLM guardrails, judges and classifiers

• Up to 100%

  evasion success against six production protection systems, including Microsoft Azure Prompt Shield and Meta Prompt Guard, using character injection and adversarial machine learning.

  Hackett et al., Lancaster University and Mindgard. arXiv:2504.11168, LLMSec 2025.

• Near random

  chance. LLM-judge performance degrades to near coin-flip under red-teaming distribution shift, across 6,642 human-verified labels. Many reported attack successes exploit judge inadequacy rather than genuine harm.

  Schwinn et al., TU Munich and Mila. arXiv:2603.06594, February 2026.

• Over 85%

  attack success rate against agentic coding assistants when adaptive strategies are used. Most defense mechanisms achieve under 50% mitigation. The authors conclude the fix has to be architectural, not filtering-based.

  Maloyan and Namiot. Review of 78 studies and 42 attack techniques. arXiv:2601.17548, January 2026.

• Up to 40%

  swing in judge robustness caused by nothing more than the choice of prompt template. Tested 15 attack methods, 7 defenses and 12 models, and found live vulnerabilities in a production deployment.

  Li et al. RobustJudge framework. arXiv:2506.09443, revised November 2025.

This is the whole argument in one line

Every language model shares one weakness: language. Put a model in the enforcement path and the defense inherits the attack surface of the thing it defends.

All four findings are publicly available preprints or conference papers. Titles, authors, venues and identifiers are given above so you can read them yourself. Verified 5 August 2026.

The vendors are not hiding this. They are saying it out loud.

What is striking is that the frontier labs, the ones with the strongest commercial reason to claim the problem is handled, are the most candid about it.

A 1% attack success rate, while a significant improvement, still represents meaningful risk. No browser agent is immune to prompt injection, and we share these findings to demonstrate progress, not to claim the problem is solved.

Robustness to adversarial attacks is a long-standing challenge for machine learning and AI, making this a hard, open problem.

Successful baseline defenses like Spotlighting or Self-reflection became much less effective against adaptive attacks.

The UK's National Cyber Security Centre put it more bluntly than any of them, in a technical note published in December 2025 titled, with some feeling, Prompt injection is not SQL injection:

It's very possible that prompt injection attacks may never be totally mitigated in the way that SQL injection attacks can be. The best we can hope for is reducing the likelihood or impact of attacks.

And OWASP, whose co-chair Steve Wilson framed it in a way worth sitting with when the 2026 LLM Top 10 landed on 3 August, with prompt injection at number one for the third consecutive year:

Prompt injection may ultimately be more like death and taxes: something organizations must continuously manage rather than expect to eliminate.

So what follows from that?

Here is where we part company with a lot of the commentary. The usual conclusion drawn from this evidence is despair, or a shrug, or a pivot to talking about governance frameworks. We think the conclusion is narrower and more useful.

If the probabilistic component cannot be made reliable, stop putting it in the position where reliability is the requirement.

An enforcement decision has exactly one property that matters: given the same input, it produces the same output, every time, and you can read why afterwards. Nothing about that requires intelligence. It requires determinism.

Compiled pattern sets do it. Entropy analysis does it. Exponential moving average drift detection does it. Structural heuristics about what a request is shaped like do it. None of these are clever. That is the point. There is no prompt you can write that talks a regular expression out of matching, because a regular expression does not have a concept of being talked to.

The obvious objection, and it is a fair one, is that deterministic rules cannot catch novel attacks. This is worth taking seriously rather than waving away.

The answer has two parts. First, the regular expression is the mechanism, not the library. The threat library is alive, hot-swapping signatures from OWASP, CrowdSec and the National Vulnerability Database with zero downtime. A static list from 2024 would indeed be useless. Nobody is proposing one.

Second, and more importantly, signature matching is one layer of thirteen. Behavioral layers do not look for strings at all. Entropy analysis notices that a payload has statistical properties normal text does not. Drift detection notices that a session's pattern has changed in a way that sessions do not normally change. A novel attack still has to act in a particular way to achieve anything, and acting is observable even when the words are new.

Where the AI does belong

Not nowhere. Just not in the enforcement path.

Models are extremely good at generating hypotheses about what a strange request might be trying to do. They are good at summarizing a week of traffic. They are good at proposing that a new pattern looks like a variation on a known technique. All of that is valuable, and all of it is survivable when it is wrong, because being wrong produces a bad suggestion rather than an unblocked attack.

So in our architecture the AI tiers sit beside the request path and never inside it. A copy of the traffic goes to them, out of band, non-blocking. They can propose a detection rule. A deterministic validator and a human being have to approve it before it becomes live policy. Switch every AI tier off and the enforcement engine is provably byte-identical, which is a property you can verify rather than a promise you have to accept.

Two ways to build the enforcement path

The industry default, and the one AI-Sentinel took

The industry default: a model judges the model

• Request

  client
• LLM judge

  probabilistic
• Target model

  probabilistic

Same class of system, same attack surface

Jailbreak the judge with the same technique you would use on the model.

AI-Sentinel: math judges, the model never does

• Request

  client
• T0 deterministic

  compiled RegexSets
• Target model

  probabilistic

Different class of system, no shared attack surface

There is no prompt that talks a regular expression out of matching.

Property	Model-based enforcement	Deterministic enforcement
Can it be jailbroken by a prompt?	Yes, demonstrated in published research	No mechanism for it to be
Same output for the same input?	Not guaranteed	Guaranteed and machine-checkable
Typical added latency	Tens to hundreds of milliseconds	2 to 8 ms measured
Can a human read the reason it blocked?	An inference, after the fact	A rule ID, in the log, at the time

“Jailbroken” claims for model-based enforcement are drawn from the published research listed in the LLM-judge evidence figure. AI-Sentinel latency figures are v6.0 internal benchmarks, June 2026.

There is a second cost, and your engineers will find it first

Even if the model-based guardrail worked perfectly, it would still be paying a latency tax on every request for the rest of its deployment.

Meta publishes measured latency for its Prompt Guard 2 classifiers: 19.3 milliseconds for the 22 million parameter version, 92.4 milliseconds for the 86 million parameter version. NVIDIA published first-party measurements for a NeMo guardrail stack, and three rails, content moderation plus jailbreak detection plus topic control, add roughly 530 milliseconds to total response time.

That is about 27 times the cost of a small encoder classifier, and orders of magnitude more than a compiled pattern match. Notably, most of the major platform products, Cloudflare, AWS Bedrock Guardrails, Azure Prompt Shields, Prisma AIRS and Cisco AI Defense, publish no latency figures at all. You are welcome to draw your own conclusion about why.

Here is what we have watched happen at three different clients. A security layer adds a few hundred milliseconds. Product notices. An exemption gets granted for the latency-sensitive path. The exemption becomes the default. Six months later the control is nominally deployed and functionally bypassed, and the only person who knows is whoever last read the routing config.

Security your engineers route around is not security. It is logging with extra steps.

The eight-second question, again

You do not need to take a position on any of this to use it. You need one question, asked in every vendor evaluation, and a willingness to sit through the answer.

What component makes the decision to block, and can that component be talked out of its job using the same technique that would work on my production model?

If the answer is yes, the vendor may still be worth buying. Plenty of these products catch plenty of attacks. But you now know the failure mode you are accepting, and you can price it, layer around it and tell your board about it honestly. Which is more than most buyers in this category currently have.

Do not take our word for any of it. Run it against your own traffic and see.

‹ All articles

Do not take our word for it. Run it against your own traffic.

Monitor mode goes in non-blocking, in minutes, with zero risk to live workflows. Within days you get a written audit of the prompt injections, extraction attempts and data leaks your current stack is not catching. Then you decide.

The audit is free. The blind spot is not.