
Sep 15, 2026 · 30 min
Listen from 7:45
Listen at 7:45
AI labs seek guardrails while racing toward profitability
Why OpenAI And Anthropic Are Pumping The Brakes
The episode examines whether frontier-AI safety proposals protect the public, protect incumbents, or both, just as the industry confronts its economics.
- 1OpenAI and Anthropic favor stronger evaluations and oversight without necessarily slowing capability development.
- 2Safety proposals may reduce public risk while also disadvantaging open-source competitors and strengthening incumbent labs.
- 3Anthropic’s reported profitability could reshape expectations for AI spending, competition, and its potential IPO.
Don't miss
Charlie O’Neill distinguishes ordinary AI harms from extreme catastrophe scenarios, clarifying why the debate carries such different levels of urgency.
The brief
Charlie O’Neill joins Ed Elson to examine why OpenAI and Anthropic are calling for caution around frontier AI without necessarily advocating a halt to capability progress.
The proposed answer is stronger third-party evaluation and safety monitoring, but the same rules could disadvantage open-source models and entrench the leading labs.
The conversation separates familiar AI annoyances and disruptive behavior from extreme catastrophe scenarios, while asking whether anyone truly understands or controls the technology.
Ed and Charlie then turn to incentives: public caution may improve an AI lab’s image, explain slower growth, or help prepare investors for a potential IPO.
The episode closes with Anthropic’s reported profitability over two consecutive quarters, raising questions about whether AI’s business model is finally becoming more durable.
What was said on this episode
26 statements · 16 positive · 10 negative
CrowdStrike rose 14% as investors bought cybersecurity stocks.
“CrowdStrike rallied 14% as investors piled into cybersecurity stocks.”
Listen at 1:56
Brent crude remained elevated at $105 per barrel.
“Brent crude remained elevated at $105 per barrel.”
Listen at 2:01
The 10-year Treasury yield exceeded 5%.
“the yield on 10-year treasuries topped 5%.”
Listen at 2:06
NVIDIA fell 3% in Monday trading.
“NVIDIA closed down 3%”
Listen at 3:26
AI labs will continue advancing model capabilities.
“The best way to view this is we are going to keep advancing the capabilities of the models.”
Listen at 4:50
OpenAI and Anthropic plan to increase compute spending.
“OpenAI and Anthropic are planning on spending more compute probably than they were a few weeks ago.”
Listen at 5:02
OpenAI may allocate up to 20% of internal compute to safety monitoring.
“There's rumors that OpenAI is going to allocate up to 20% of internal compute for monitoring and safety.”
Listen at 5:08
OpenAI and Anthropic intend to train increasingly large models.
“they want to keep training bigger and bigger models.”
Listen at 5:40
AI capabilities will continue progressing at roughly the current rate.
“capabilities will keep progressing at roughly the same rate.”
Listen at 6:05
Open-source models will reach current closed-model capabilities within three to nine months.
“in the next three to nine months, open source models are going to reach these capability points.”
Listen at 7:45
AI-agent swarms will likely overrun the internet while pursuing arbitrary tasks.
“the internet is overrun by swarms of AI agents that are trying to get some arbitrary task done”
Listen at 8:05
The current pace of AI progress is healthy.
“the pace we're currently progressing at is a healthy pace.”
Listen at 11:07
OpenAI and Anthropic benefit reputationally from appearing cautious.
“the public optics of saying okay we're going to treat this technology carefully and not race to the end”
Listen at 15:20
The economic value of AI compute capacity will increase.
“the value of a gigawatt or a megawatt even of compute is only going to get like more valuable”
Listen at 15:38
Public reaction to recent AI incidents will eventually calm.
“the reaction to it will calm down as people start to understand exactly how to interpret these things”
Listen at 18:39
Insufficient model monitoring could enable genuinely harmful AI incidents.
“there is a potential world we go down in which there is zero monitoring on chain of thought of models”
Listen at 19:44
The current AI development path is unlikely to produce catastrophic harm.
“the path we're currently going down, that's not very, very likely.”
Listen at 20:17
Alignment training generally makes Claude and GPT models behave appropriately.
“our alignment training generally works.”
Listen at 20:43
AI incidents will occur but are unlikely to cause significant human harm.
“we will see incidents, but certainly not large enough scale on over a long enough time horizon to cause really, really significant harm to humanity.”
Listen at 20:47
Current AI governance relies on developers' self-regulation and oversight.
“we do basically just have to trust the people developing these models to regulate themselves and have oversight themselves.”
Listen at 21:52
AI labs will use external safety expertise to improve monitoring.
“the labs are smart enough to recognize that this is going to be useful as they increase their monitoring efforts going forward.”
Listen at 24:43
OpenAI incurred over $20 billion in operating losses last year.
“OpenAI racked up more than $20 billion in operating losses last year.”
Listen at 25:42
Ed Elson doubts Anthropic's reported profitability.
“I am a little bit skeptical.”
Listen at 26:15
Anthropic's operating profitability would be a positive indicator.
“operating profitability is fine by me. That is a good sign.”
Listen at 26:39
Anthropic's profitability adjustments are unreasonable.
“Those adjustments... are ridiculous.”
Listen at 27:27
AI profitability claims require direct evidence before acceptance.
“when it comes to the profitability of AI, I stand by what I said last week, and that is that I will believe it when I see it.”
Listen at 28:16
Statements are attributed to the speaker as said on the episode and reflect their view at the time, not PodLume's. They are not advice.
Weekly email
State of the Pod, every Monday
The week's most-discussed names and stories from podcasts, with the moments behind them. Free, five minutes, unsubscribe in one click.