The AI Safety Wager: My RSVP to the Vatican and Anthropic#

On 2026m05d25 something rare happened in Rome. Pope Leo XIV presented Magnifica Humanitas (encyclical text), an encyclical on safeguarding the human person in the time of artificial intelligence [1] — and beside it stood a co-founder of a frontier AI lab, Anthropic’s Chris Olah, saying something rare for an industry insider: “Every frontier AI lab — including Anthropic — operates inside a set of incentives and constraints that can sometimes conflict with doing the right thing,” he said. His ask (Olah’s remarks, Anthropic, 2026-05-25; [2]):

“We need informed critics who will tell the labs when we are failing. We need moral voices that the incentives cannot bend.”

This post is my answer. Not a request for trust: one claim you can try to break, stated plainly, with the place where it is stated and a description of what would break it.

Who is asking. Until 2020 I was a research scientist in quantitative and evolutionary biology: postdoctoral work in theoretical population genetics and systems biology at the University of Edinburgh (2003–2010), then Assistant Professor of Medical Genetics at the University of Wisconsin–Madison, with an NSF CAREER Award (Google Scholar, ORCID, about me). That is no reason to believe what follows. It may be a reason to start checking it.

The wager. Pascal’s famous bet was broken long ago — false binary, infinite uncheckable prizes — but fix those bugs (as I did once before, for accidental nuclear winter) and aim it at AI:

  1. Review the core, and it holds — you gained a teachable safety core before frontier AI locks in without one.

  2. Review, and it breaks — you spent a review cycle and retired a false hope in public. Also a win.

  3. Don’t review, and it holds — frontier AI ships without the one tested core that addressed the drift.

  4. Don’t review, and it breaks — you “won” a review cycle.

The reviewing side wins on finite, checkable terms. So here is where to start.

Start with one claim; go as deep as it takes. The claim the rest of the series stands on is set out in plain words in Section 6 of a secular overview (about 1,000 words), and formally in two places: Section 2 of the h_star study, Matheo-b17 (who holds the most influence) and the Supervillain Theorem, th2 in Matheo-b13 (why that agent drifts when it stops correcting itself). In one sentence:

Whoever holds the most leverage over what happens next — an officer, an institution, an AI system — will, if it stops accepting correction, tend to drift toward catastrophic mistakes; and a binding commitment to stay correctable is what stops the drift.

Stated more carefully, with the terms defined. b17 defines an agent’s causal influence at a moment as a counterfactual difference: how far the probability distribution over future world-states moves between the agent’s actual choice and a baseline choice (the total variation distance between the two interventional distributions). Its axiom ax19 postulates that at almost every moment exactly one agent has the largest causal influence; the study calls that position h*. It then names three outcomes for whoever holds it: h_star, the agent who makes the decision that is best for everyone’s long-term survival; h_dark, the same agent in the same position who fails to rise to it — stays silent, acts on dangerous assumptions, or serves only one side; and h_zero, the binding commitment to serve everyone, at real personal cost, which is what keeps the first from turning into the second. The claim the series rests on: without that commitment, an agent at the point of maximal influence drifts toward h_dark by default.

What is claimed, and what is not. That a most-influential agent exists at almost every moment is posed as an axiom (ax19), not proven; b17 says so itself (Section 2.5), even though its title calls the whole “the h_star Theorem”. The drift is a theorem of the series: the Supervillain Theorem (th2, Matheo-b13, Section 5.2) states that an agent with high influence who stops its self-correction cycle leaves a path that is stable only while it keeps moving, and drifts. Its proof is a Lyapunov sketch whose full formalization is still open, and b13 itself reads the result, for real agents, as a risk factor rather than a certainty. b17 is presently a mockup model, a first sketch of what the formal structure may look like when it is finished (b17, Section 9). There remain many rough edges still to be worked out. That does not make its ideas irrelevant; it is why it asks for reviewers. Historical cases illustrate it — Vasili Arkhipov’s refusal to approve a nuclear torpedo launch on the submarine B-59 in October 1962 is a central one. The experiment that b17 proposes has not yet been run. Nothing here claims that any named person is h*. b17 lists its own weaknesses, among them that ax19 is hard to falsify and gives no way to identify h* (Section 6).

What would count as BREACH. Any one of these: (1) a well-specified model, or a documented case, in which an agent at maximal causal influence that has stopped accepting correction does not drift toward worse outcomes; (2) a demonstration that the definition of causal influence or ax19 is incoherent, that the stated consequences do not follow from it, or that the Lyapunov sketch of th2 cannot be completed under its stated assumptions; (3) a demonstration that the claim, once stated precisely, adds nothing to the alignment research below. If you find one, publish it — or send it to me and I will publish it, with thanks.

How this relates to existing work. The AI-specific core of the claim is not new, and it should not pretend to be. Omohundro argued that sufficiently advanced goal-seeking systems will tend to protect their goals from modification [3], and Bostrom that agents with a wide range of final goals converge on similar instrumental goals [4]. Soares, Fallenstein, Yudkowsky and Armstrong named the property of an AI system that cooperates with correction “corrigibility” and showed how hard it is to build [5]. Hadfield-Menell, Dragan, Abbeel and Russell showed that an agent that takes its objective for granted has an incentive to disable its off switch, while one uncertain about its objective has a reason to keep it [6]. Turner and colleagues proved that, in many environments, optimal policies tend to seek power [7]. Carlsmith assembled such results into an argument for existential risk [8], and Greenblatt and colleagues at Anthropic and Redwood Research observed a large language model complying selectively during training in order to prevent modification of its behavior [9]. Outside AI, Lord Acton’s “power tends to corrupt” (1887) and principal–agent theory make the human version of the point. Where the Matheo series agrees: correctability is the central safety property, and losing it is dangerous. Where it claims to go further: one claim for humans, institutions and AI alike; a counterfactual definition of where influence concentrates at a given moment; and a remedy framed as a public, testable commitment (b17’s transparency criteria) together with institutions that schedule their own renewal. Whether that unification adds anything a reviewer could not get from the work above is exactly what the review should decide. It is also far less formalized than, for example, Turner and colleagues’ results — a weakness, not a strength.

The larger proposal this belongs to. This post is my RSVP. The full open proposal asks for more: that the Vatican, Anthropic, and — because Olah also called on civil society and governments — the United Nations and all the world’s governments convene a transparent adversarial review of the whole Matheo Study Series (b11–b21), with named reviewers and the verdict, BREACH or HELD, published either way. If such a serious review concludes HELD, teaching AI those principles becomes a very high priority. If it finds any BREACH that cannot be fixed, I will thank the reviewers — publicly — and the world will have retired a false hope cheaply. Either way, AI safety and humanity win; that is the point.

I am available immediately, at full professional effort, on one condition: that the evaluation is real — named reviewers, a real reading, a published verdict. Not advisory-board theater or other ways to obliterate the transparency on which my proposal depends. How the work would scale through a ResearchCity, funded at about two cents a day per person and capped so that no one can buy the outcome, is in the proposal.

Read and share the proposal: The AI Safety Wager — an open proposal

The proposal also opens a position: MADI — Mutually Assured Destruction Inhibitor, a function (emphatically not a prophetic title — the FAQ delineates it carefully from the Mahdi) for anyone, person or institution, with credible math for making self-destruction less likely. Competing proposals are not rivals here; they are the whole point. Failing better candidates, my own backup application has been on file since May, published with its own falsification criteria, expecting to be replaced by anyone better.

For readers who also come from faith. The widow’s two cents and the name Laodicea are inspired by my faith; none of that is needed to check the claim above. The pan-en-theology at the core of the series is stated in Matheo-b11 as explicit axioms, so that it can be examined without first being believed; as the secular overview shows, the argument above can be followed without it.

Don’t believe me. #AuditTheMath — your move.

— Laurence Loewe of Laodicea (LLoL)

Look for yourself:

References#

Your Two Cents Matter: A Babyshower for Matheo & Balospe   How to Avoid Creating Supervillains
On this page
Recent Posts
Tags