Tomorrow, Today
Tomorrow, Today

Nov 13, 2026 · 32 min

Autonomous AI agents expose a new layer of safety risk

The Hidden Risks of Autonomous AI Agents | Dustin Allen x Shekhar Natarajan

As AI systems take on longer workflows and interact with one another, small model errors can compound into failures involving privacy, money, and human control.

3 key takeaways
  1. 1Probabilistic agents become harder to govern as multi-step workflows compound individually manageable errors.
  2. 2Test-driven governance, deterministic inference, and oracle-guided distillation offer safeguards without requiring companies to harvest personal data.
  3. 3Open models can improve privacy and customization, but agent-to-agent standards and human autonomy remain unresolved.

Don't miss

Dustin Allen uses an HR agent changing an employee’s bank account to show how safeguards must span the model, payroll, and enterprise systems.

The brief

Dustin Allen of Trinitite and Shekhar Natarajan frame autonomous agents as a governance problem: probabilistic models become difficult to control when they operate across long workflows or in swarms.

The proposed answer is test-driven governance: evaluate expected outcomes before deployment, use deterministic inference to expose edge cases, and distill fixes into runtime-swappable low-rank adapters.

An HR agent changing an employee’s bank account makes the stakes concrete, showing how malformed outputs and downstream integrations can turn reliable steps into a dangerous workflow.

The conversation weighs open-weight models against closed systems, arguing that privacy and customization matter while acknowledging the governance burden of giving agents more autonomy.

The closing question is larger than AI safety: if machines take over more manual work, how can people preserve privacy, agency, and a distinctly human future?

Weekly email

State of the Pod, every Monday

The week's most-discussed names and stories from podcasts, with the moments behind them. Free, five minutes, unsubscribe in one click.

Listen to the full episode and explore every guest, topic, and moment on PodLume.