Onnex
Your business has the answer.
Let Onnex find it.
LOADING
. . .
Completed
0

Blog

Who Owns the AI Risk? Bridging the Gap Between Vulnerability and Responsibility

272 AI experts just ranked the 24 ways AI can hurt you. The most useful finding in the paper isn't a risk. It's the gap where the owner should be.

In June a team led by MIT FutureTech and the University of Queensland published a three round Delphi study of 272 international AI experts. Over September to November 2025, those experts rated 24 AI risks on four things: how severe, how likely, who is exposed, and who is responsible.

Most of the coverage will quote the headline number. Under a business as usual trajectory, the panel put at least a 10% chance of catastrophic harm on 18 of the 24 risks inside five years (p. 2). Catastrophic is defined in the rubric as more than 1 million deaths, or more than $100B in losses, or civilizational scale intangible damage.

Those numbers will get attention, but it’s not the part you should care about.

The part you should care about is on page 13.

Everyone is exposed. Two actors are responsible. The middle is empty.

The experts were asked to rate seven actors on vulnerability, then rate the same seven on responsibility. The answers do not line up.

Figure 1: Vulnerability was diffuse. Responsibility was concentrated. Redrawn from median ratings reported in Saeri et al. 2026, pp. 11-13.

Vulnerability was diffuse. AI users and affected stakeholders scored 4 to 5 out of 5 across nearly all 24 risks, with consensus. Responsibility was concentrated. General purpose AI developers and governance actors (governments, regulators, standards bodies) scored 4 to 5 across nearly all 24 risks, also with consensus (pp. 12 to 13).

So the people getting hurt have no lever, and the people with the lever are two steps removed from the harm. The authors say this plainly: “those who are most responsible for addressing AI risks are not those who are most vulnerable, leading to misaligned incentives” (p. 4).

Then they name the failure mode. Shared responsibility can produce “an accountability sink, where responsibility shared across many actors becomes responsibility held by none” (p. 19).

Sit with that for a second, because it describes most enterprise AI programs I see right now. The model vendor says the model is aligned. The cloud says the tenant is isolated. The security team says the WAF is up. Legal says the DPA is signed. Every layer is doing something. Nobody can show you what their layer stopped.

One expert broke ranks, and he was right

Buried in the supplementary comments on risk 2.2, AI system security vulnerabilities and attacks, one panelist pushed back on the consensus that model developers own this:

The model is not where the vulnerability is, it is in the AI system. Model deployment must engage in security posture management that mitigates known and unknown vulnerabilities. Putting the responsibility with model developers is wrong-headed, as no system is expected to secure itself.

Delphi participant, Supplementary File 9, p. S31

Another, commenting on cyberattacks, was more specific about where the job actually sits:

Responsibility follows control of attack and actuation surfaces and the ability to remediate. Deployers are primary because they operate keys, CI or CD, networks, and incident response.

Delphi participant, Supplementary File 9, p. S37

Those two comments are the operational reading of the whole study. Frontier labs can restrict capabilities and secure weights. Regulators can set mandatory controls. Neither of them is holding your API keys, your RAG index, your agent tool permissions, or your pager. You are.

Why this is a separate discipline, not a new module on your firewall

The same section contains the cleanest statement of the architectural problem I have read in an academic paper:

Due to an inherent systemic architectural flaw, where all prompts are processed identically regardless of whether they originate from the ‘admin’ or the ‘user’, mitigation can only be achieved through external safeguards.

Delphi participant, Supplementary File 9, p. S31

Figure 2: Instruction and data arrive in the same channel. There is no boundary to enforce, which is why the enforcement has to sit outside the model.

Read that again. A language model has no privilege boundary. Instruction and data arrive in the same channel and get the same treatment. That is not a bug in one vendor's model. It is a property of the architecture.

Your WAF does not have this problem, because your WAF is not a model. It also cannot fix this problem, for exactly the same reason. It has no semantic view of what is being asked of your model, or what your model is about to say back. Conventional cybersecurity was not built for a system whose control plane is English.

That is why the study's taxonomy carves out AI system security vulnerabilities as its own subdomain (2.2), separate from every conventional category. The panel is treating this as a distinct class. So should you.

The same expert adds that external guardrails are themselves error prone. That is fair, and it is the sharpest question in the field. It is also an argument about which kind of external safeguard you pick. A safeguard that makes probabilistic judgments inherits the failure mode it was hired to catch. A deterministic one does not. If you are going to put something outside the model, put something outside the model's failure class too.

The number that changes the board conversation

On page 17 the authors do something most risk papers avoid. They translate their probabilities into governance language:

Under many risk-governance frameworks, a 10% probability of catastrophic outcome over five years would be considered “intolerable”, likely triggering mandatory mitigation requirements.

Saeri et al. 2026, p. 17

Figure 3: Eighteen of 24 risk domains cleared the threshold that standard risk-governance frameworks treat as intolerable.

Eighteen of 24 risks clear that bar under business as usual. The paper anchors this to the frameworks your auditors already use, NIST AI RMF and ISO 31000, both cited on page 5.

If you run a risk register, that is not an abstract debate about AI doom. That is a category of risk sitting above your own tolerance threshold with no named owner and no evidenced control. Auditors notice that.

The good news is in the paper's own second scenario

The experts rated everything twice. Once under business as usual, and once under what the authors call Pragmatic Mitigations, defined simply as a world where “organizations & governments make pragmatic and cost-effective efforts to address risks from AI” (p. 9).

Figure 4: Severity fell for all 24 risks, significant across the board with large effect sizes (r = 0.73 to 0.84, p. 10).

Severity fell for all 24 risks. Every one. Mean reductions of 0.36 to 0.53 points on a 5 point scale, statistically significant across the board with large effect sizes (p. 10). Weapons and cyberattacks dropped 9 percentage points. Dangerous capabilities dropped 10.

Note what that scenario does not require. It does not require a treaty, a moratorium, or a breakthrough in interpretability. It requires organizations doing the cheap, obvious, unglamorous things. The paper even names some of the mechanisms: “input filtering, output classification” and model weight security, described as system level mitigations that reduce likelihood or severity “across many risks at once” (pp. 17 to 18).

Inspecting what goes into your models and what comes back out is not exotic. It is the pragmatic mitigation. It has been sitting in the too hard basket because nobody was assigned it.

Where it lands first

Figure 5: Weapons and cyberattacks was the single most selected top-three concern across all 272 experts (p. 16).

If you want to know whether this is your problem yet, the sector data is on page 14. Experts rated Information and National Security as most vulnerable across all risks. Finance and Insurance was rated similarly high specifically for fraud and scams, AI system security vulnerabilities, and AI system safety failures, reflecting in one panelist's words “both direct attack vectors (fraud, market manipulation) and regulatory exposure from AI system failures.” Health Care scored high on privacy, discrimination, and overreliance.

The thing worth taking away

Defense in depth and an accountability sink look identical from the outside. Same diagram. Same layers. Same vendors on the slide.

Figure 6: The diagram is not the control. The evidence is.

The only thing that separates them is whether each layer can show you what it caught.

The study makes the case for distributed responsibility as a strength, “where each actor implements safeguards” (p. 18). That only holds if the safeguards are real and someone checked. Otherwise you have four vendors, one diagram, and nobody who can answer the auditor's question.

Two hundred and seventy two experts told you the risk sits above tolerable, told you the mitigations work, and left the middle of the chart empty.

Stop defaulting to ‘sharing the risk’ and therefore basically doing nothing.

Own your piece and prove it works.

--------------------------------------------------------------------------------

Source: Saeri, A. K., Graham, J., Noetel, M., Slattery, P., Thompson, N., et al. (2026). “Prioritization of Risks from Artificial Intelligence: A Delphi Study of 272 International Experts.” arXiv:2606.04490v1. Data at osf.io/pj2qr. Risk taxonomy from the MIT AI Risk Repository (Slattery et al.). Study funded by Commonwealth Bank of Australia, who reviewed the design but did not influence collection, analysis, interpretation or reporting.

All charts are original work by AI Sentinel, redrawn from values stated in the text of the paper. No source artwork is reproduced.

This article was drafted using artificial intelligence tools to assist with content generation, and subsequently reviewed, heavily edited, and fact-checked by a human editor to ensure accuracy, quality, and originality.