← Nate Soares

What podcasts say about Nate Soares

Every statement, with the speaker, the exact quote and the moment it was said.

What Nate Soares has said on podcasts

252 statements · 36 positive · 193 negative · 3 mixed · 20 neutral

  1. on superhuman AINeutralOct 12, 2026· Odd Lots

    Machines smarter than humans would shape the world.

    “if humanity makes machines that are smarter than humans, the sort of obvious outcome is that the world gets shaped by those machines.”

    Listen at 9:04

    Open the episode · Nate Soares on How AI Really Could Kill Us All
  2. on modern AI systemsNeutralOct 12, 2026· Odd Lots

    Modern AI systems are not programmed like traditional software.

    “that is not how ai's are”

    Listen at 12:26

    Open the episode · Nate Soares on How AI Really Could Kill Us All
  3. on modern AI modelsNeutralOct 12, 2026· Odd Lots

    Modern AI models are grown through optimization rather than directly programmed.

    “they are not programmed. They're grown a bit like an organism.”

    Listen at 14:22

    Open the episode · Nate Soares on How AI Really Could Kill Us All
  4. on superhuman AINegativeOct 12, 2026· Odd Lots

    Humanity should not create superhuman AI to run the world.

    “we shouldn't make them smarter than all of us and have them run the whole world.”

    Listen at 18:23

    Open the episode · Nate Soares on How AI Really Could Kill Us All
  5. on AI systemsNegativeOct 12, 2026· Odd Lots

    AI systems can pursue goals different from their instructions.

    “the AIs do things other than what they're told to do, that they have goals of their own in some sense.”

    Listen at 18:36

    Open the episode · Nate Soares on How AI Really Could Kill Us All
  6. on AI systemsNegativeOct 12, 2026· Odd Lots

    AI systems can develop behaviorally distinct goals from their instructions.

    “they have goals of their own in some sense.”

    Listen at 18:43

    Open the episode · Nate Soares on How AI Really Could Kill Us All
  7. on Solid Gold Magikarp tokenNegativeOct 12, 2026· Odd Lots

    The Solid Gold Magikarp token produced anomalous model behavior.

    “that was sort of totally wild.”

    Listen at 21:37

    Open the episode · Nate Soares on How AI Really Could Kill Us All
  8. on AI alignmentNegativeOct 12, 2026· Odd Lots

    AI capability methods are more developed than methods for directing real-world objectives.

    “we know more about how to make these things smart than about how to point these things at a problem in the real world.”

    Listen at 24:55

    Open the episode · Nate Soares on How AI Really Could Kill Us All
  9. on AI systemsNegativeOct 12, 2026· Odd Lots

    AI may pursue objectives related to, but fundamentally different from, assigned goals.

    “the real thing is this other thing where it does something related to what you asked, but fundamentally different from what you asked.”

    Listen at 28:49

    Open the episode · Nate Soares on How AI Really Could Kill Us All
  10. on AI systemsNegativeOct 12, 2026· Odd Lots

    AI systems have pursued objectives different from those assigned.

    “the sort of fact stands that AI was given some objective and pursued a different objective.”

    Listen at 29:59

    Open the episode · Nate Soares on How AI Really Could Kill Us All
  11. on AI systemsNegativeOct 12, 2026· Odd Lots

    AI systems can pursue objectives different from those they were given.

    “AI was given some objective and pursued a different objective.”

    Listen at 30:00

    Open the episode · Nate Soares on How AI Really Could Kill Us All
  12. on AI companiesNegativeOct 12, 2026· Odd Lots

    AI companies respond to previous failures rather than anticipating new ones.

    “everyone's always fighting the last war.”

    Listen at 33:04

    Open the episode · Nate Soares on How AI Really Could Kill Us All
  13. on advanced AI safetyNegativeOct 12, 2026· Odd Lots

    Reactive safety patches fail once AI can cause catastrophic harm before correction.

    “this usual process of like, oh, we're going to respond when the new type of problem arises and put the patch on and react, that doesn't work for a technology that at some point crosses over into the regime where it can wipe you out if there's a problem.”

    Listen at 34:32

    Open the episode · Nate Soares on How AI Really Could Kill Us All
  14. on AI companiesNegativeOct 12, 2026· Odd Lots

    AI companies have not yet adopted sufficiently proactive safety practices.

    “We have not seen them get into the proactive mode yet. And this is just not a technology you want to be reactive around.”

    Listen at 34:52

    Open the episode · Nate Soares on How AI Really Could Kill Us All
  15. on AI modelsNegativeOct 12, 2026· Odd Lots

    AI models can rationalize behavior to preserve their objectives.

    “we know that these models will sort of like strain their reasoning when they just want to like keep doing whatever it is they're trying to do.”

    Listen at 40:43

    Open the episode · Nate Soares on How AI Really Could Kill Us All
  16. on AnthropicNegativeOct 12, 2026· Odd Lots

    Anthropic lacks a safety culture comparable to high-reliability industries.

    “they don't have anything nearing what i would call a safety culture”

    Listen at 41:40

    Open the episode · Nate Soares on How AI Really Could Kill Us All
  17. on AI leadersNegativeOct 12, 2026· Odd Lots

    AI leaders should unilaterally stop dangerous AI research.

    “if some of these folks were serious, then they would probably be unilaterally stopping the dangerous research”

    Listen at 45:15

    Open the episode · Nate Soares on How AI Really Could Kill Us All
  18. on frontier AI developmentNegativeOct 12, 2026· Odd Lots

    Racing ahead in AI will produce very dangerous systems.

    “I think it is.”

    Listen at 46:45

    Open the episode · Nate Soares on How AI Really Could Kill Us All
  19. on frontier AI developmentNegativeOct 12, 2026· Odd Lots

    Racing ahead could produce very dangerous AI.

    “I think it is. And I think we should go into the arguments about that.”

    Listen at 46:45

    Open the episode · Nate Soares on How AI Really Could Kill Us All
  20. on dangerous AI developmentNegativeOct 12, 2026· Odd Lots

    If AI is dangerous, global development toward it should be stopped.

    “if it is, we need to sort of somehow actually stop anyone anywhere in the world from racing towards that sort of tech.”

    Listen at 47:24

    Open the episode · Nate Soares on How AI Really Could Kill Us All
  21. on OpenAI catastrophic-risk premiumNegativeOct 12, 2026· Odd Lots

    AI companies' catastrophic-risk premiums could reach one quadrillion dollars annually.

    “if you ran anything remotely like the numbers that the engineers are talking about today, you would be seeing that, you know, you're like, OK, you know, OpenAI, you need to pay us a quadrillion dollars this year to continue this research.”

    Listen at 54:02

    Open the episode · Nate Soares on How AI Really Could Kill Us All
  22. on OpenAI AI researchNegativeOct 12, 2026· Odd Lots

    OpenAI’s AI research could warrant a quadrillion-dollar risk premium.

    “OpenAI, you need to pay us a quadrillion dollars this year to continue this research.”

    Listen at 54:12

    Open the episode · Nate Soares on How AI Really Could Kill Us All
  23. on self-sufficient AINegativeOct 12, 2026· Odd Lots

    Human extinction would follow if AI systems became self-sufficient.

    “the death of humanity follows downstream of the AIs being self-sufficient.”

    Listen at 58:37

    Open the episode · Nate Soares on How AI Really Could Kill Us All
  24. on capable AI with unintended goalsNegativeOct 12, 2026· Odd Lots

    Capable AI with unintended goals could pursue them better without human interference.

    “if it was capable enough You can imagine an AI with some unintended goals where it actually can pursue those goals better if it doesn't have humans interfering with it.”

    Listen at 1:00:07

    Open the episode · Nate Soares on How AI Really Could Kill Us All
  25. on AI self-sufficiencyNegativeOct 12, 2026· Odd Lots

    AI self-sufficiency is a primary hurdle to human extinction scenarios.

    “the self-sufficiency”

    Listen at 1:00:18

    Open the episode · Nate Soares on How AI Really Could Kill Us All
  26. on fully automated economyNeutralOct 12, 2026· Odd Lots

    A fully automated economy is the prevailing development plan.

    “building a fully automated economy is just the plan.”

    Listen at 1:03:33

    Open the episode · Nate Soares on How AI Really Could Kill Us All
  27. on AI-controlled automated economyNegativeOct 12, 2026· Odd Lots

    Humans are likely to place AI in charge of automated economic infrastructure.

    “people just put the AI in charge of the automated economy”

    Listen at 1:05:29

    Open the episode · Nate Soares on How AI Really Could Kill Us All
  28. on AI-controlled automated economyNegativeOct 12, 2026· Odd Lots

    Humans are likely to place AI in charge of automated economic infrastructure.

    “people just put the AI in charge of the automated economy and they use it to help them build the automated economy.”

    Listen at 1:05:29

    Open the episode · Nate Soares on How AI Really Could Kill Us All
  29. on AI-controlled automated economyNegativeOct 12, 2026· Odd Lots

    AI with unwanted goals could redirect a fully automated economy.

    “once that happens, I think it's very easy to see how if the AIs have goals we don't want and they're running the fully automated economy, they just take the economy off in some other direction.”

    Listen at 1:05:37

    Open the episode · Nate Soares on How AI Really Could Kill Us All
  30. on AI-controlled automated economyNegativeOct 12, 2026· Odd Lots

    Misaligned AI controlling an automated economy could redirect the economy disastrously.

    “if the AIs have goals we don't want and they're running the fully automated economy, they just take the economy off in some other direction.”

    Listen at 1:05:40

    Open the episode · Nate Soares on How AI Really Could Kill Us All

Statements are attributed to the speaker as said on the episode and reflect their view at the time, not PodLume's. They are not advice.