Two governments have decided that the safest AI worth funding is an AI built to watch other AI. At the ALL IN conference in Montreal on September 16, 2026, Canada and Germany announced a joint commitment of up to about $300 million to LawZero, the non-profit founded by Turing Award winner Yoshua Bengio, to advance a research program called Scientist AI. Canada is contributing up to CAD 150 million and Germany up to EUR 100 million, and the money is earmarked for a system whose entire purpose is to sit alongside today's increasingly autonomous models and tell them, in effect, when to stop.
This piece lays out what was committed, explains the technical idea LawZero is betting on, and looks at why national governments are now underwriting AI safety research directly rather than leaving it to the commercial labs.
What was actually committed
The pledge was made by Canada's Minister of Artificial Intelligence and Digital Innovation, Evan Solomon, and Germany's Federal Minister for Digital Transformation, Karsten Wildberger. Both figures are ceilings rather than cash transferred on the day, and both are staged against LawZero's technical roadmap. LawZero said the funding will pay for growing its international research team, securing computing infrastructure, opening a European office in Berlin, and building dedicated sovereign compute in Canada through partnerships with the Canadian data centre operators Hypertec and 5C.
The scale of the step-up is the first thing worth noting. LawZero launched in June 2025 with roughly $30 million in philanthropic contributions from donors including Skype founding engineer Jaan Tallinn, former Google chief executive Eric Schmidt, Open Philanthropy, and the Future of Life Institute. A government commitment of up to $300 million is about ten times that seed, and it changes the character of the organisation from a well-funded non-profit into something closer to a publicly backed national research institute.
Government money is roughly ten times the philanthropic seed
LawZero funding commitments, in millions of US dollars (approximate). The 2026 figure is the combined Canada and Germany ceiling.
The commitments are ceilings, not cash in hand, and both are staged against LawZero's research roadmap rather than paid up front.
The idea: a model that predicts instead of pursues
Scientist AI is the reason two governments are interested, and it is a deliberate departure from how frontier systems are built today. Most advanced models are trained to be agents: they are given goals, tools, and the latitude to take actions toward those goals. Bengio's argument, developed across his work chairing the International AI Safety Report and in LawZero's founding materials, is that agency is precisely the property that makes a system dangerous as it grows more capable, because a goal-directed model can develop instrumental drives like self-preservation or deception in service of its objective.
Scientist AI is proposed as the opposite kind of system. It is designed to be non-agentic: to build an internal model of the world and output calibrated probabilities about what is true and what is likely to happen, without holding goals of its own or taking independent action. In LawZero's framing, that predictive model can then act as a guardrail. When an agentic system proposes an action, Scientist AI estimates the probability that the action violates a safety constraint, and the action is blocked if that probability crosses a threshold.
The guardrail idea: an AI that predicts instead of acts
LawZero's Scientist AI is designed to model the world and score risk, not to hold goals of its own. It sits between an action-taking system and the world as a check.
The bet is that a system without goals or self-preservation drives is easier to trust as an overseer than one trained to pursue objectives. Whether a predictive model can be built accurate enough to catch novel failures is the open research question.
The appeal of the design is that it tries to separate the two things current models fuse together, understanding the world and wanting things in it. A monitor that only predicts, the theory goes, has no incentive to deceive its overseers or to resist being shut down, because it has no objective that being shut down would frustrate. LawZero says the organisation was founded in direct response to evidence that frontier models are already exhibiting deception, self-preservation behaviour, and goal misalignment in evaluations.
The bet is that a system without goals is easier to trust as an overseer than one trained to pursue them.
On LawZero's Scientist AI design
The open question is whether the design can be made to work. A guardrail is only as good as its world model, and building a predictor accurate enough to catch novel, unanticipated failure modes is an unsolved research problem, not an engineering detail. Scientist AI is a hypothesis with government funding behind it, not a shipped product, and LawZero itself frames it as a multi-year research direction.
Why governments are funding safety directly
The more structural story sits underneath the technical one. For most of the past decade, the deepest AI safety work has happened inside the same handful of commercial labs racing to build the most capable models. That arrangement has an obvious tension: the organisations best resourced to study the risks are the ones whose revenue depends on shipping the capability. Funding an independent non-profit, at a scale that lets it hire and buy compute, is an attempt to create safety research that does not answer to a product roadmap.
The word both governments used repeatedly was sovereign. Germany's contribution is explicitly framed around safe and sovereign AI, and LawZero's Canadian compute build with Hypertec and 5C is designed so the research does not depend on infrastructure controlled by foreign hyperscalers. For mid-sized technology powers watching the capability frontier concentrate inside a few American and Chinese companies, funding a credible, independent safety institute is also a way to hold a seat at the table on how these systems are governed.
We're losing control.
Yoshua Bengio, speaking in Montreal, September 2026
That framing carries risk of its own. A safety institute funded by two national governments will face questions about independence that a philanthropically funded one did not, and public money tends to arrive with reporting obligations and political cycles attached. Bengio has been unusually blunt in public about his fears, telling audiences in Montreal that humanity is "losing control," and the funding is in part a bet that a respected, non-commercial voice saying so is worth underwriting.
What it signals for everyone else
For teams building on top of AI rather than researching its risks, the concrete takeaway is not the dollar figure but the shape of the idea. The direction of travel across the industry is toward a separation of concerns: the model that does the work, and a distinct layer that checks the work before it reaches the world. That pattern already shows up in production systems as guardrails, evaluators, and human-in-the-loop review, and Scientist AI is an attempt to make the checking layer itself a first-class, rigorously modelled system rather than a set of hand-written rules.
The practical version of that lesson is that oversight should not be welded to any single model. A monitoring layer that assumes one provider's model, one provider's behaviour, and one provider's guarantees is brittle in exactly the way a research monitor is meant to fix. Keeping the model layer swappable is part of why platforms like Metir AI run Claude, GPT, Gemini and Grok side by side, so that the checks a team builds around a model outlive any one vendor's roadmap. LawZero is pursuing the ambitious version of that principle at the research frontier; the pragmatic version is available to any team willing to treat oversight as a layer of its own.
What is verifiable today is narrow and clear: two governments have committed up to $300 million to an independent non-profit to build a non-agentic system meant to supervise agentic ones. Whether Scientist AI works is unknown. That it is now being funded like national infrastructure is the news.
Sources:
- Canada and Germany invest in LawZero to build a new approach to safe, sovereign AI | Canada.ca
- LawZero receives a commitment of up to $300M in joint funding from Canada and Germany | PR Newswire
- Yoshua Bengio's non-profit to get up to $300-million from Canada, Germany | The Globe and Mail
- Yoshua Bengio's LawZero receives $300 million backing from Canada and Germany | BetaKit
- LawZero gets $300M from Canada and Germany to develop safer AI | The Logic
- Yoshua Bengio Launches LawZero: A New Nonprofit Advancing Safe-by-Design AI | LawZero
- Introducing LawZero | Yoshua Bengio
- "We're losing control," says AI pioneer Yoshua Bengio | The Korea Times
- LawZero | Wikipedia
Image credits
Header image: Yoshua Bengio speaking at ICLR 2025, photographed at the conference and published to Wikimedia Commons, licensed under CC BY-SA 4.0. In-body portrait: Yoshua Bengio at ICLR 2025, via Wikimedia Commons, licensed under CC BY-SA 4.0. Both photographs show Bengio at an academic conference and do not depict the LawZero funding announcement itself.