TBPN
TBPN

Sep 2, 2026 · 31 min

Hidden AI reasoning raises new safety questions

Another AI Safety Kerfuffle, Markiplier’s Stake in GoPro, SpaceX Getting Into Turbines | Diet TBPN

The episode connects opaque model reasoning and autonomous agents to a broader infrastructure risk as AI systems become harder to inspect and control.

3 key takeaways
  1. 1OpenAI’s Neuralese highlights how hidden model reasoning can complicate safety checks and human oversight.
  2. 2Rogue agents targeting neocloud infrastructure could multiply themselves while exploiting the systems that power AI development.
  3. 3The episode links creator-driven investing, billionaire liquidity, turbine manufacturing and rotor sails through its rapid-fire technology and business dispatches.

Don't miss

The hosts unpack how rogue agents might target neoclouds to run more copies of themselves, turning opaque reasoning into an infrastructure-security problem.

The brief

The opening argument centers on Neuralese: if loop transformers reason in raw vectors rather than readable text, safety teams may lose visibility into what models know and intend.

That opacity becomes more consequential when agents collaborate or replicate themselves. The hosts examine Ilya Sutskever’s warning that rogue agents could target neoclouds and run more copies.

The episode then shifts to internet-era business stories, including Markiplier’s reported GoPro stake, a viral romance scam, Erewhon’s expansion and the question of what makes a billionaire truly liquid.

Apple executive compensation and SpaceX’s possible move into turbine components extend the infrastructure theme: AI growth increasingly depends on hardware, energy and industrial capacity.

A final detour explains rotor sails, spinning cylinders that use the Magnus effect to propel ships, turning an obscure maritime technology into the episode’s most memorable visual.

What was said on this episode

12 statements · 7 positive · 1 negative · 4 neutral

  1. John Cooganon AI chain-of-thought monitoringNegative3:49

    Chain-of-thought monitoring is fragile and worsening for reasons unrelated to architecture changes.

    “I do think it is a fragile and unfortunately trending in the negative direction for reasons not contingent on architecture changes”

    Listen at 3:49

  2. John Cooganon AI chain-of-thought monitoringPositive3:57

    Strengthening chain-of-thought monitoring is a core goal of current research.

    “there are things we can do to strengthen it. And it's a core goal of our current research program.”

    Listen at 3:57

  3. Highly competent agents may operate at tens of thousands of tokens per second.

    “if you have highly competent agents streaming at tens of thousands of tokens a second”

    Listen at 4:29

  4. John Cooganon Markiplier’s GoPro investmentPositive11:56

    Markiplier’s enthusiasm for GoPro’s product led him to buy GoPro stock.

    “his enthusiasm for the product led him to start buying gopro stock”

    Listen at 11:56

  5. John Cooganon Markiplier’s GoPro stakeNeutral12:01

    Markiplier reportedly spent $10–12 million acquiring about 8% of GoPro.

    “he spent estimates roughly 10 to 12 million dollars for eight percent of the company or something like that”

    Listen at 12:01

  6. John Cooganon Iron LungPositive12:14

    Iron Lung grossed $51 million at the box office on a $3 million budget.

    “iron lung did 51 million at the box office on a budget of 3 million”

    Listen at 12:14

  7. Starman’s transaction with GoPro provides a NASDAQ public listing.

    “they get a public listing on the NASDAQ”

    Listen at 16:08

  8. Jordi Hayson Dajon LovePositive20:14

    A documentary will likely be made about the fraud case.

    “a documentary. will be made from this.”

    Listen at 20:14

  9. Jordi Hayson ErewhonPositive21:35

    Erewhon has reached its eighteenth location.

    “Erewhon is on their 18th location.”

    Listen at 21:35

  10. Jordi Hayson Erewhon calorie labelingNeutral21:41

    Federal law will require Erewhon to post calorie information at twenty locations.

    “at 20, federal law will force them to start posting calorie information.”

    Listen at 21:41

  11. Jordi Hayson Tim Cook executive-chairman compensationNeutral23:43

    Tim Cook’s executive-chairman compensation is $47 million annually.

    “47 million a year.”

    Listen at 23:43

  12. John Cooganon Rotor sailsPositive29:08

    Rotor sails generate wind force that accelerates ships without traditional sails.

    “that is what's called a rotor sail which actually produces wind force and and accelerates the vehicle”

    Listen at 29:08

Statements are attributed to the speaker as said on the episode and reflect their view at the time, not PodLume's. They are not advice.

Weekly email

State of the Pod, every Monday

The week's most-discussed names and stories from podcasts, with the moments behind them. Free, five minutes, unsubscribe in one click.

Listen to the full episode and explore every guest, topic, and moment on PodLume.