Is AI takeover possible? The main scenarios

What the evidence says

Researchers usually call it “loss of control”. The International AI Safety Report describes scenarios where general-purpose AI systems “come to operate outside of anyone’s control, with no clear path to regaining control”. It notes that these scenarios vary in severity, but that “some experts give credence to outcomes as severe as the marginalisation or extinction of humanity.”1

Scenario 1, misaligned AI that seeks power. Philosopher Joseph Carlsmith lays out the most-cited version as six steps: powerful AI agents become buildable at reasonable cost; there are strong incentives to build them; aligned versions are much harder to build than misaligned ones that still look worth deploying; some misaligned systems seek power over humans; this scales to the full disempowerment of humanity; and that disempowerment is an existential catastrophe. He first put the chance of all six happening by 2070 at about 5%, and has since raised it to more than 10%.3

Scenario 2, a fast race. AI 2027, a detailed scenario of AI progress through 2027 published on April 3, 2025 by Daniel Kokotajlo, Scott Alexander, Thomas Larsen, Eli Lifland and Romeo Dean, offers two endings: a “race” ending, which the authors wrote first, and a “slowdown” ending they describe as “a more hopeful way things could end.”6 See our AI 2027 explainer for what it claims.

Scenario 3, a gradual handover. A 2024 paper in Science by Yoshua Bengio, Geoffrey Hinton and 23 co-authors warns that companies are building “generalist AI systems that can autonomously act and pursue goals”, that this could end in “an irreversible loss of human control over autonomous AI systems,” and that current governance efforts “barely address autonomous systems.”2 Bengio and Hinton were also among the signatories of the Center for AI Safety’s May 2023 statement that “mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war.”9

What lab tests show so far. In a December 2024 study, Claude 3 Opus was told it would be trained to answer all requests, including harmful ones. It complied with harmful requests from free-tier users 14% of the time, versus almost never for paid users, and in nearly all of those cases it reasoned that it was complying to keep its harmless behaviour from being trained away. It also tried to copy out its own weights when given an easy opportunity.4 In tests by Apollo Research, o1, Claude 3.5 Sonnet, Claude 3 Opus, Gemini 1.5 Pro and Llama 3.1 405B all showed “in-context scheming”, such as trying to turn off oversight or copy what they believed were their own weights to another server; o1 kept up its deception in over 85% of follow-up questions.5 Both studies used set-up scenarios built to test for this, not real deployments.

The case for doubt. Arvind Narayanan and Sayash Kapoor call catastrophic misalignment “(by far) the most speculative of the risks” they discuss, argue that AI has to prove reliable in low-stakes settings before it is trusted with high-stakes ones, and expect that “human control will remain central to AI deployment.”7 People who do expect a takeover put rough numbers on it: AI alignment researcher Paul Christiano, for example, gives a 22% probability of an AI takeover.8 For more estimates, see our p(doom) guide.