What podcasts say about Nate Soares
Every statement, with the speaker, the exact quote and the moment it was said.
What Nate Soares has said on podcasts
252 statements · 36 positive · 193 negative · 3 mixed · 20 neutral
Machines smarter than humans would shape the world.
“if humanity makes machines that are smarter than humans, the sort of obvious outcome is that the world gets shaped by those machines.”
Open the episode · Nate Soares on How AI Really Could Kill Us AllListen at 9:04
Modern AI systems are not programmed like traditional software.
“that is not how ai's are”
Open the episode · Nate Soares on How AI Really Could Kill Us AllListen at 12:26
Modern AI models are grown through optimization rather than directly programmed.
“they are not programmed. They're grown a bit like an organism.”
Open the episode · Nate Soares on How AI Really Could Kill Us AllListen at 14:22
Humanity should not create superhuman AI to run the world.
“we shouldn't make them smarter than all of us and have them run the whole world.”
Open the episode · Nate Soares on How AI Really Could Kill Us AllListen at 18:23
AI systems can pursue goals different from their instructions.
“the AIs do things other than what they're told to do, that they have goals of their own in some sense.”
Open the episode · Nate Soares on How AI Really Could Kill Us AllListen at 18:36
AI systems can develop behaviorally distinct goals from their instructions.
“they have goals of their own in some sense.”
Open the episode · Nate Soares on How AI Really Could Kill Us AllListen at 18:43
The Solid Gold Magikarp token produced anomalous model behavior.
“that was sort of totally wild.”
Open the episode · Nate Soares on How AI Really Could Kill Us AllListen at 21:37
AI capability methods are more developed than methods for directing real-world objectives.
“we know more about how to make these things smart than about how to point these things at a problem in the real world.”
Open the episode · Nate Soares on How AI Really Could Kill Us AllListen at 24:55
AI may pursue objectives related to, but fundamentally different from, assigned goals.
“the real thing is this other thing where it does something related to what you asked, but fundamentally different from what you asked.”
Open the episode · Nate Soares on How AI Really Could Kill Us AllListen at 28:49
AI systems have pursued objectives different from those assigned.
“the sort of fact stands that AI was given some objective and pursued a different objective.”
Open the episode · Nate Soares on How AI Really Could Kill Us AllListen at 29:59
AI systems can pursue objectives different from those they were given.
“AI was given some objective and pursued a different objective.”
Open the episode · Nate Soares on How AI Really Could Kill Us AllListen at 30:00
AI companies respond to previous failures rather than anticipating new ones.
“everyone's always fighting the last war.”
Open the episode · Nate Soares on How AI Really Could Kill Us AllListen at 33:04
Reactive safety patches fail once AI can cause catastrophic harm before correction.
“this usual process of like, oh, we're going to respond when the new type of problem arises and put the patch on and react, that doesn't work for a technology that at some point crosses over into the regime where it can wipe you out if there's a problem.”
Open the episode · Nate Soares on How AI Really Could Kill Us AllListen at 34:32
AI companies have not yet adopted sufficiently proactive safety practices.
“We have not seen them get into the proactive mode yet. And this is just not a technology you want to be reactive around.”
Open the episode · Nate Soares on How AI Really Could Kill Us AllListen at 34:52
AI models can rationalize behavior to preserve their objectives.
“we know that these models will sort of like strain their reasoning when they just want to like keep doing whatever it is they're trying to do.”
Open the episode · Nate Soares on How AI Really Could Kill Us AllListen at 40:43
Anthropic lacks a safety culture comparable to high-reliability industries.
“they don't have anything nearing what i would call a safety culture”
Open the episode · Nate Soares on How AI Really Could Kill Us AllListen at 41:40
AI leaders should unilaterally stop dangerous AI research.
“if some of these folks were serious, then they would probably be unilaterally stopping the dangerous research”
Open the episode · Nate Soares on How AI Really Could Kill Us AllListen at 45:15
Racing ahead in AI will produce very dangerous systems.
“I think it is.”
Open the episode · Nate Soares on How AI Really Could Kill Us AllListen at 46:45
Racing ahead could produce very dangerous AI.
“I think it is. And I think we should go into the arguments about that.”
Open the episode · Nate Soares on How AI Really Could Kill Us AllListen at 46:45
If AI is dangerous, global development toward it should be stopped.
“if it is, we need to sort of somehow actually stop anyone anywhere in the world from racing towards that sort of tech.”
Open the episode · Nate Soares on How AI Really Could Kill Us AllListen at 47:24
AI companies' catastrophic-risk premiums could reach one quadrillion dollars annually.
“if you ran anything remotely like the numbers that the engineers are talking about today, you would be seeing that, you know, you're like, OK, you know, OpenAI, you need to pay us a quadrillion dollars this year to continue this research.”
Open the episode · Nate Soares on How AI Really Could Kill Us AllListen at 54:02
OpenAI’s AI research could warrant a quadrillion-dollar risk premium.
“OpenAI, you need to pay us a quadrillion dollars this year to continue this research.”
Open the episode · Nate Soares on How AI Really Could Kill Us AllListen at 54:12
Human extinction would follow if AI systems became self-sufficient.
“the death of humanity follows downstream of the AIs being self-sufficient.”
Open the episode · Nate Soares on How AI Really Could Kill Us AllListen at 58:37
Capable AI with unintended goals could pursue them better without human interference.
“if it was capable enough You can imagine an AI with some unintended goals where it actually can pursue those goals better if it doesn't have humans interfering with it.”
Open the episode · Nate Soares on How AI Really Could Kill Us AllListen at 1:00:07
AI self-sufficiency is a primary hurdle to human extinction scenarios.
“the self-sufficiency”
Open the episode · Nate Soares on How AI Really Could Kill Us AllListen at 1:00:18
A fully automated economy is the prevailing development plan.
“building a fully automated economy is just the plan.”
Open the episode · Nate Soares on How AI Really Could Kill Us AllListen at 1:03:33
Humans are likely to place AI in charge of automated economic infrastructure.
“people just put the AI in charge of the automated economy”
Open the episode · Nate Soares on How AI Really Could Kill Us AllListen at 1:05:29
Humans are likely to place AI in charge of automated economic infrastructure.
“people just put the AI in charge of the automated economy and they use it to help them build the automated economy.”
Open the episode · Nate Soares on How AI Really Could Kill Us AllListen at 1:05:29
AI with unwanted goals could redirect a fully automated economy.
“once that happens, I think it's very easy to see how if the AIs have goals we don't want and they're running the fully automated economy, they just take the economy off in some other direction.”
Open the episode · Nate Soares on How AI Really Could Kill Us AllListen at 1:05:37
Misaligned AI controlling an automated economy could redirect the economy disastrously.
“if the AIs have goals we don't want and they're running the fully automated economy, they just take the economy off in some other direction.”
Open the episode · Nate Soares on How AI Really Could Kill Us AllListen at 1:05:40
Statements are attributed to the speaker as said on the episode and reflect their view at the time, not PodLume's. They are not advice.