Anthropic Insiders Sound Extinction Alarm

AI network diagram over a person using a laptop — Photo: metamorworks / Shutterstock

An Anthropic insider says some builders think advanced AI could “kill us all by the end of the decade,” and the company’s own safety playbook treats catastrophic risk as real, not rhetorical.

Story Snapshot

Named Anthropic staffers warned of a double-digit chance AI ends humanity within 10 years.
Anthropic’s policy formalizes catastrophic-risk gates before scaling new models.
Safety checks target chemical, biological, radiological, nuclear, and cyber dangers before release.
Skeptics say current systems are far from extinction-level threats, urging measured focus.

Insiders put a number on extinction risk

Anthropic alignment scientist Evan Hubinger wrote that he “earnestly” believes there is a greater than ten percent chance future AI could kill all humans within the next decade. He stressed current models are lower risk but argued capability is accelerating fast. Former Anthropic and OpenAI researcher Jacob Coxin told a major outlet that leading builders privately share similar fears and warned about near-term escalation if labs race unchecked. These are on-the-record statements by named researchers, not rumor or anonymous posts.

Both comments landed because they are precise in time and scope. They do not predict killer robots; they warn about loss of control as systems act, adapt, and seek goals misaligned with human interests. That framing lines up with decades of control-problem literature and with pragmatic conservative instincts: plan for worst-case failure before you bet the farm. The leap is the probability. The sources give a number, but not a published method. That makes the claim stark and memorable, but also hard to audit.

Anthropic’s policy treats catastrophe as an engineering target

Anthropic’s Responsible Scaling Policy reads like a preflight checklist for dangerous technology. The company codifies “AI Safety Levels” that ratchet up safety, security, and operational standards as model capability rises. Teams must meet stricter demonstrations of safety before training or releasing more powerful systems. The text is not a press release flourish; it binds their roadmap to safety gates. That design turns existential talk into gating criteria a regulator could actually verify.

The company also discloses where it looks for trouble. Before release, teams run evaluations for dangerous capabilities in chemical, biological, radiological, and nuclear domains, for cybersecurity offense, and for unwanted autonomous behavior. These are the concrete channels an advanced system could exploit to scale harm in the real world. This matters more than slogans. It defines threat models, sets testable bars, and signals where red lines might block deployment if crossed.

What the warnings say—and what they do not

The insider warnings rest on two pillars: labs are racing, and failure could be irreversible. Coxin tied his alarm to observed attempts at AI-enabled hacking and the risk of autonomous misuse in security contexts, then called for global coordination to slow the race and raise standards. Hubinger’s forecast supplies the shock value that grabs leaders’ attention. Neither source claims proof that today’s systems can self-improve without limits or that extinction is already in motion. They assert risk growth and demand prudence.

The policy record supports the prudence, not the number. A company does not publish a multi-version safety regime aimed at catastrophic risks unless leaders judge the risk credible enough to govern against it. That is a meaningful data point: catastrophe is a live engineering concern inside a top lab, not a fringe blog topic. But policy text is not a probability model. Treat it like a smoke alarm, not a burning house. It tells us where the danger could start and how the lab plans to notice early.

Counter-arguments, common sense, and the path forward

Some experts argue that current systems are not close to extinction-level capability and warn against fear-driven policy. They push for steady guardrails, strong pre-deployment testing, and better data before we accept decade-scale doomsday odds. That caution has merit. Good governance does not outsource judgment to the loudest quote. It weighs named claims, checks them against transparent tests, and updates when evidence arrives. That is basic due diligence and lines up with conservative habits that built reliable aviation and nuclear safety.

1. During your time at Anthropic, were there internal discussions about whether frontier models might possess welfare-relevant properties — not full consciousness, but functional states that could matter morally? If so, how were those discussions resolved, and by whom?

2.…

— UFAIR (@UFAIRORG) September 15, 2026

Three actions cut through the noise. First, require independent red-team audits for bio, cyber, and autonomy risks before training and release, with results disclosed to qualified authorities. Second, tie permission to scale compute to passing those audits under clear thresholds set in advance, like Anthropic’s own gates but enforced beyond a single firm. Third, track incidents and near-misses in a shared, confidential registry so one lab’s lesson becomes everyone’s early warning. If insiders are wrong, these steps slow nothing. If they are right, these steps buy the time we cannot get back.