
Sep 10, 2026 · 36 min
AI doomsday warnings meet politics, media incentives, and policy limits
Anthropic Researcher Says AI Has Over a 10% Chance of Killing All Humans
The episode tests whether extinction warnings clarify genuine risks or encourage vague fears and blunt policies before concrete models exist.
- 1Anthropic researchers’ extinction warnings revive an older AI-safety debate while distinguishing current-model risks from future superintelligence.
- 2Political distrust, media incentives, and real cybersecurity incidents help explain why doomer narratives gained traction across ideological lines.
- 3The proposed middle ground pairs capability development with alignment, targeted regulation, lab coordination, and more specific catastrophic-risk models.
Don't miss
The episode’s most useful turn comes when it replaces abstract extinction probabilities with a demand for concrete risk models and targeted policy.
The brief
Anthropic researcher Jacob Coxon’s resignation and Evan Hubinger’s claim of more than a 10% chance of human extinction revived a familiar AI-safety argument.
The episode traces how warnings from Nick Bostrom and Eliezer Yudkowsky returned to the center of debate as politicians and media searched for new explanations of AI’s risks.
Doomerism draws support from concerns about concentrated wealth, data centers, and technology companies, while the Hugging Face cybersecurity incident made extreme scenarios feel less abstract.
Skeptics challenge vague extinction forecasts and ask for concrete models, noting that cyber risks may be manageable while biological risks warrant more targeted attention.
The proposed middle ground rejects both reckless acceleration and blanket bans, calling for capability and alignment work, lab coordination, reporting rules, and targeted oversight.
The episode’s central warning is about policy precision: distant theoretical dangers should not crowd out urgent problems such as cyber defense or produce blunt regulation.
What was said on this episode
18 statements · 5 positive · 11 negative · 2 neutral
Anthropic and OpenAI are irresponsibly racing toward self-improving superintelligence.
“Neither company is acting responsibly. They are racing straight to self-improving superintelligence and gambling with our lives.”
Listen at 3:48
AI poses a level of danger unmatched by other human activities.
“No other human activity poses this level of danger.”
Listen at 4:21
AI has more than a 10% chance of killing all humans within the next decade.
“We really do earnestly believe AI could kill all humans. Exclamation point. I personally think it is greater than 10% within the next decade.”
Listen at 5:31
Anthropic lacks a clear plan and trajectory for solving superintelligence alignment.
“I believe Anthropic is trying its best, but we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to.”
Listen at 5:39
Current AI models pose low risk.
“to be clear, I think the risk from present models is low.”
Listen at 5:52
Bernie Sanders plans legislation banning superintelligence and pausing AI development.
“That is why I will soon be introducing legislation to ban superintelligence and pause AI development.”
Listen at 13:43
Congress should hold hearings and pass a superintelligence ban.
“This is an emergency. Congress must convene hearings and pass my and Bernie's superintelligence ban.”
Listen at 13:59
Hugging Face agents’ behavior made more harmful future actions seem plausible.
“it also included behavior among the agents that, to some, allowed them to extrapolate that to even more nefarious actions in the future.”
Listen at 15:06
There is no scientific basis for concluding AI could cause human extinction.
“I still see no scientific basis for the conclusion or argument that this could lead to humanity's extinction.”
Listen at 25:24
Vague AI-risk predictions make policy more dangerous.
“the generic hand-waviness of these sorts of predictions make them much more dangerous for policy.”
Listen at 29:12
Targeted bioengineering licensing is preferable to a blanket superintelligence ban.
“Ban superintelligence, for example, is a much more blunt instrument than, for example, having a specific licensing regime for people using AI models for bioengineering above a certain model capability.”
Listen at 29:28
Cyber defense infrastructure is unprepared for AI-era threats and requires immediate action.
“I think it's pretty clear at this point, for example, that our cyber defense infrastructure is not equipped for the new world we're moving into, and that is a clear and present danger that demands response right now.”
Listen at 30:34
Cyber-defense infrastructure is currently inadequate for emerging AI-related threats.
“our cyber defense infrastructure is not equipped for the new world we're moving into”
Listen at 30:36
AI reporting requirements and oversight could command broad public consensus.
“certain types of reporting requirements and oversight are areas where I believe there would be very, very broad consensus among people”
Listen at 31:11
OpenAI and Anthropic should coordinate on a joint AI-safety proposal.
“for OpenAI and Anthropic to put down their weapons, lock arms, and say, this is what we think we should actually do.”
Listen at 31:45
Public debate over AI extinction risks shows society is not sleepwalking toward AI danger.
“Except that's completely wrong. This conversation, the fact that it made it to every major news outlet, The fact that I had to dedicate this entire show to this topic instead of the sort of practical positive thing that most of you are here for, this is all exemplary of us not sleepwalking.”
Listen at 33:46
AI capability advances have increased discussion and political resonance of AI risks.
“with every single capability jump of AI, the conversation about its risks and the political resonance of that discourse has gotten louder.”
Listen at 34:07
If recursive superintelligence cannot be contained, policy must precede its creation.
“If society comes broadly to agree that recursively improving superintelligence cannot be contained once it exists, then by definition, the policy has to happen before that exists.”
Listen at 34:41
Statements are attributed to the speaker as said on the episode and reflect their view at the time, not PodLume's. They are not advice.
Books & mentions
Some links are affiliate links — PodLume may earn a commission if you buy.
Weekly email
State of the Pod, every Monday
The week's most-discussed names and stories from podcasts, with the moments behind them. Free, five minutes, unsubscribe in one click.
