TBPN
TBPN

Sep 16, 2026 · 31 min

AI evaluators face a trust and accountability test

METR and AI Regulation, Zuck Pushes Back on AI Slowdown, Fed Hikes Rates | Diet TBPN

The episode connects AI safety oversight to legal responsibility, monetary policy, and the incentives shaping frontier-lab transparency.

3 key takeaways
  1. 1Independent AI evaluators must distinguish forecast risks from documented model behavior and safety violations.
  2. 2Zuckerberg’s safety posture favors continued AI development and practical safeguards over deeper existential-risk arguments.
  3. 3Higher rates and contested compute data expose the financial and political infrastructure behind the AI build-out.

Don't miss

The hosts contrast broad calls for independent AI evaluation with Anthropic’s more concrete offer of badge, Slack, and desk access to evaluators.

The brief

John Coogan and Jordi Hays open on whether independent AI evaluators should forecast future catastrophe or inspect models, logs, and concrete safety violations.

The hosts argue that AI oversight inherits nuclear regulation’s accountability problem: evaluators need independence, while governments and labs must decide who bears responsibility for autonomous cyber incidents and economic damage.

A 25-basis-point Federal Reserve hike and hawkish outlook sharpen the discussion, exposing AI businesses and investment styles that relied on near-zero borrowing costs.

Zuckerberg’s call to move quickly while building safely sounds like ordinary product-safety practice to the hosts, leaving the deeper existential-risk debate unresolved.

The sharpest contrast comes over transparency: Anthropic’s reported willingness to give evaluators badge, Slack, and desk access is treated as more consequential than generic calls for outside review.

The episode closes on Kalshi’s AI compute tracker, reportedly ordered removed by the Commerce Department, and what that controversy reveals about measuring the infrastructure race.

What was said on this episode

19 statements · 9 positive · 5 negative · 1 mixed · 4 neutral

  1. Jordi Hayson Frontier model behavior expertiseNegative2:05

    Few groups have deeply studied frontier model behavior for a long period.

    “there are not that many groups that have been this invested in understanding frontier model behavior for this long”

    Listen at 2:05

  2. John Cooganon AI regulatory evaluationPositive2:37

    Regulators can train qualified hires to evaluate AI systems, incidents, risks, and liabilities.

    “you can just tell the regulator, if it's someone, you don't need to tell, like if you hire someone and they are able to get up to speed on how these systems work and evaluate them. and look at logs of different incidents, see what's happening, assess the risk level, assess the liabilities.”

    Listen at 2:37

  3. John Cooganon U.S. Navy nuclear trainingPositive2:59

    Navy enlistees can become responsible for submarine nuclear-plant security within 18 months.

    “you can be 22 years old, not graduate from college, go into the Navy, enlist in the Navy, not even in the officer's training program. And in 18 months, you can be responsible for the security of a nuclear power plant on a submarine.”

    Listen at 2:59

  4. Taking superforecasting seriously can produce substantial financial gains.

    “super forecasting, super important, awesome. Also just like fun read, really cool. And if you take it seriously, you make a lot of money.”

    Listen at 4:15

  5. Jordi Hayson MeterNegative5:12

    Meter’s financing and history undermine trust in its independence.

    “the financing structure, the history of the various parties there just makes it so that people don't have any trust that they would act like that.”

    Listen at 5:12

  6. John Cooganon Nuclear regulationNeutral6:05

    Scientists advised nuclear regulators rather than directly serving as regulators.

    “the scientists didn't actually wind up being. the regulators, they wound up being the advisors.”

    Listen at 6:05

  7. John Cooganon AI agent swarmsNegative8:52

    Agent swarms can perpetrate cybersecurity violations without directly causing economic losses.

    “with particularly with like agent swarms perpetrating cyber security violations that don't directly cause economic harm”

    Listen at 8:52

  8. John Cooganon Technology businessesNegative14:26

    Some technology businesses only work when interest rates are zero.

    “this whole tech thing, it only works when the interest rates are zero.”

    Listen at 14:26

  9. John Cooganon PipeNegative15:12

    Pipe’s low-rate borrowing-and-lending model breaks down when early-stage borrowing costs rise.

    “they're basically borrowing at 0%, then they're lending to a company at 5% or something. Whatever their spread is, it's actually justifiable to an earlier piece of the market. But that sort of breaks down when you have to go to a company and say, hey, you want money at 12% or something like that for an early stage company?”

    Listen at 15:12

  10. Jordi Hayson Meta product safetyPositive18:30

    Meta must ensure its products are safe and secure for billions of users.

    “This is what Meta has been doing forever. You have to do it. You have billions of users. You got to make sure they're safe and secure.”

    Listen at 18:30

  11. Jordi Hayson AI companiesNeutral18:58

    AI-product companies already engage in these safety practices.

    “every single company that makes AI products already does this.”

    Listen at 18:58

  12. Jordi Hayson Mark Zuckerberg’s AI risk beliefsPositive20:57

    Zuckerberg likely assigns zero probability to AI doom and high probability to abundance.

    “he has a P doom of zero and his P abundance is high.”

    Listen at 20:57

  13. John Cooganon AI capability developmentMixed22:10

    Coding capability development is aligned with recursive self-improvement; image generation is not.

    “the race to become really, really good at coding is super aligned with RSI. The race to do image generation is not.”

    Listen at 22:10

  14. John Cooganon Independent AI evaluator accessPositive24:41

    Allowing nonemployee evaluators broad access and whistleblowing authority is a major step.

    “the prospect of bringing in a non-employee who is there explicitly to whistleblow effectively and like is allowed to talk about anything and like is is you know third party evaluator like that is a huge step”

    Listen at 24:41

  15. John Cooganon AnthropicNeutral25:20

    Anthropic-style evaluator access is unusual and newly emerging.

    “this is sort of unprecedented. This doesn't happen a lot. Like this is new.”

    Listen at 25:20

  16. Jordi Hayson MusePositive26:53

    Meta’s Muse product is receiving excellent reviews and appears impressive.

    “Muse is getting amazing reviews. It seems like an amazing product.”

    Listen at 26:53

  17. John Cooganon Kalshi AI compute price trackerPositive27:10

    Kalshi’s AI compute tracker could proxy the scale of AI infrastructure build-out.

    “this will allow you to understand, like, basically a proxy for the AI build-out.”

    Listen at 27:10

  18. John Cooganon AI compute costsPositive29:40

    AI compute costs are now a highly important economic indicator in the United States.

    “The cost of compute has become one of the most important numbers in the U.S. economy.”

    Listen at 29:40

  19. John Cooganon Older AI chipsNeutral29:45

    Older AI chips collateralize billions of dollars in neocloud borrowing.

    “older chips, which serve as collateral for billions of dollars of borrowing by neoclouds”

    Listen at 29:45

Statements are attributed to the speaker as said on the episode and reflect their view at the time, not PodLume's. They are not advice.

Weekly email

State of the Pod, every Monday

The week's most-discussed names and stories from podcasts, with the moments behind them. Free, five minutes, unsubscribe in one click.

Listen to the full episode and explore every guest, topic, and moment on PodLume.