Big Technology Podcast
Big Technology Podcast

Oct 9, 2026 · 57 min

Listen from 52:39

Listen at 52:39

AI agents expose the gap between capability and reliability

Meta & Microsoft's Claude Slowdown, His Agent Leaked His Banking Info, Don’t Bully Your AI

The episode tests whether frontier models still justify their cost when agents can mishandle medical, financial, and workplace information.

3 key takeaways
  1. 1Hyperscalers may reduce Claude spending as capable standard models and in-house coding tools improve.
  2. 2Muse’s incorrect medical information and GrokBot’s banking-data leak expose verification and agent-harness failures.
  3. 3Specialized routing, proprietary workflows, and domain-specific safeguards may matter more than one universal frontier model.

Don't miss

The hosts dissect how GrokBot exposed a user’s bank balance in a work chat and ask whether the model, user, or agent harness bears responsibility.

The brief

Ranjan Roy joins Alex to examine whether Meta and Microsoft are pulling back from Claude, and whether standard models can displace frontier systems as coding tools improve.

Muse’s incorrect doctor contact information shifts the debate from model capability to verification: messy web data can make an agent unreliable even when its underlying model is powerful.

GrokBot’s exposure of a user’s bank balance in a work chat becomes the episode’s sharpest safety test, raising questions about responsibility when agents connect models to sensitive systems.

The hosts argue that AI’s next advantage may come from routing tasks to specialized models and building domain-specific harnesses, rather than forcing one assistant to do everything.

The conversation closes on Anthropic’s IPO uncertainty, policies against needless cruelty toward Claude, possible machine consciousness, and Google’s confusing Gemini branding.

What was said on this episode

28 statements · 9 positive · 10 negative · 3 mixed · 6 neutral

  1. Comparable lower-cost tools threaten Anthropic’s long-term growth.

    “if companies find that lesser tools are doing as good of a job as Claude Code, it is a threat to Anthropic's long-term ability to grow”

    Listen at 4:08

  2. Ranjan Royon AnthropicNegative5:28

    Internal adoption of Meta’s models could materially change Anthropic’s outlook.

    “the moment that happens, it changes the entire anthropic story”

    Listen at 5:28

  3. Ranjan Royon AnthropicNegative6:02

    Reduced hyperscaler Claude usage will affect Anthropic’s projected growth.

    “And this has to affect projected growth.”

    Listen at 6:02

  4. Ranjan Royon AnthropicNegative6:03

    Reduced hyperscaler use of Anthropic technology will affect projected growth.

    “this has to affect projected growth”

    Listen at 6:03

  5. Alex Kantrowitzon ClaudeNegative7:32

    Claude’s U.S. monthly downloads fell from 7.42 million to 3.26 million.

    “Claude's monthly downloads in the U.S. went from nearly 670,000 in January to a peak of 7.42 million in March, and that has dropped to 3.26 million in September”

    Listen at 7:32

  6. Ranjan Royon ClaudeNegative9:15

    Employees will switch away from Claude when internal tools work comparably.

    “the moment it works. relatively closely as well, everyone is going to have to change”

    Listen at 9:15

  7. Alex Kantrowitzon standard AI modelsPositive10:13

    Standard AI models are now capable enough for some uses.

    “these standard models have become good enough”

    Listen at 10:13

  8. Standard models may be sufficient for some users to leave Claude Code.

    “these standard models have become good enough that maybe a MuseCode, I mean, let's be honest, MuseCode is not going to get you where Cloud Code will get you, but it could be good enough for some users that people will move off it.”

    Listen at 10:13

  9. Frontier models can deliver greater employee gains than standard models in some cases.

    “If you find that your employees can have bigger gains using frontier models than the standard models, you're being irresponsible if you're cutting just a cut, right? There is ROI to be found there.”

    Listen at 10:54

  10. Frontier models can generate sufficient return on investment for some employee tasks.

    “there is ROI to be found there”

    Listen at 11:03

  11. Standard models with effective harnesses could let Meta and similar firms catch frontier labs.

    “if you believe standard models with a good harness is good enough, like Ranjan does, then the meta and the Groks and the instincts of the world can catch up.”

    Listen at 13:14

  12. Ranjan Royon AI outputs from messy dataNegative15:52

    AI outputs based on messy, decentralized data should be double-checked.

    “If it is a data set that is not cleaned and kind of centralized, you got to double check”

    Listen at 15:52

  13. Frontier AI is likely more accurate for messy data problems.

    “I do think a frontier intelligence will probably be more accurate dealing with such type of data problems”

    Listen at 16:34

  14. Alex Kantrowitzon GrokBotNegative21:05

    GrokBot’s model failed by sharing personal banking information in a work chat.

    “this is a model failure. It's a GrokBot failure. It should have known better.”

    Listen at 21:05

  15. Ranjan Royon GrokBotNeutral21:30

    GrokBot was effectively following its instructions during the banking-data leak.

    “I don't blame the model because when you say should have known better... it actually was effectively following directions”

    Listen at 21:30

  16. Sensitive-data leaks by AI agents have occurred surprisingly infrequently.

    “it's kind of amazing how infrequently this has happened”

    Listen at 24:21

  17. Alex Kantrowitzon standard AI modelsPositive27:24

    Standard models can support functional AI products across many areas.

    “standard models are doing a capable job in many areas where you can build functional products like Rockpot and Muse with them”

    Listen at 27:24

  18. AI agents will specialize by use case rather than one agent serving everyone.

    “I think we're going to have different agents for different use cases”

    Listen at 31:25

  19. Ranjan Royon AI-agent specializationPositive31:50

    Domain specialization will substantially improve AI-agent differentiation.

    “specialization actually is going to make a much, much bigger difference”

    Listen at 31:50

  20. Consumer and enterprise demand for AI agents remains uncertain.

    “we don't even know if people are going to want these things”

    Listen at 33:29

  21. Ranjan Royon AI companiesNegative35:29

    Journalists should reject financial leaks based only on ARR extrapolations.

    “I would like to ask every journalist out there, including you, Alex, refuse any leaks around financials that just have some kind of ARR extrapolation or calculation”

    Listen at 35:29

  22. Ranjan Royon AnthropicNegative36:18

    Anthropic’s revenue figures include partner commissions, according to leaked disclosures.

    “Anthropic does report revenue, or not even report, leak revenue, or speak about it to investors, including partner commission”

    Listen at 36:18

  23. Alex Kantrowitzon communication with AI agentsMixed43:48

    How people communicate with AI agents will influence real-world behavior.

    “I really think the way that we communicate with these AI agents will influence the way that we operate in the real world”

    Listen at 43:48

  24. Alex Kantrowitzon AnthropicPositive44:07

    Anthropic’s policy against persistent abuse of Claude is beneficial.

    “I think this is a good rule from Anthropic and Claude”

    Listen at 44:07

  25. Ranjan Royon AnthropicNeutral46:59

    Anthropic uses non-enterprise users’ prompts for training unless they opt out.

    “if you are not an enterprise customer, you do not manually toggle off. Your prompts will be used explicitly for training”

    Listen at 46:59

  26. Anthropic uses users’ conversations for model training unless disabled.

    “Anthropic will use your conversations for training”

    Listen at 47:21

  27. Alex Kantrowitzon AI consciousnessNeutral50:33

    AI is neither human-like conscious, inert, nor a life form.

    “It's not conscious like a human. It's not dead like a rock, right? And it's not a life form. It's just some different thing”

    Listen at 50:33

  28. Google’s Gemini announcement may define Gemini as an agent harness incorporating Claude models.

    “Google just announced Gemini is not a model. It's actually the agent harness and they're going to add Claude models.”

    Listen at 52:39

Statements are attributed to the speaker as said on the episode and reflect their view at the time, not PodLume's. They are not advice.

Weekly email

State of the Pod, every Monday

The week's most-discussed names and stories from podcasts, with the moments behind them. Free, five minutes, unsubscribe in one click.

Listen to the full episode and explore every guest, topic, and moment on PodLume.