
Nov 13, 2026 · 32 min
Autonomous AI agents expose a new layer of safety risk
The Hidden Risks of Autonomous AI Agents | Dustin Allen x Shekhar Natarajan
As AI systems take on longer workflows and interact with one another, small model errors can compound into failures involving privacy, money, and human control.
- 1Probabilistic agents become harder to govern as multi-step workflows compound individually manageable errors.
- 2Test-driven governance, deterministic inference, and oracle-guided distillation offer safeguards without requiring companies to harvest personal data.
- 3Open models can improve privacy and customization, but agent-to-agent standards and human autonomy remain unresolved.
Don't miss
Dustin Allen uses an HR agent changing an employee’s bank account to show how safeguards must span the model, payroll, and enterprise systems.
The brief
Dustin Allen of Trinitite and Shekhar Natarajan frame autonomous agents as a governance problem: probabilistic models become difficult to control when they operate across long workflows or in swarms.
The proposed answer is test-driven governance: evaluate expected outcomes before deployment, use deterministic inference to expose edge cases, and distill fixes into runtime-swappable low-rank adapters.
An HR agent changing an employee’s bank account makes the stakes concrete, showing how malformed outputs and downstream integrations can turn reliable steps into a dangerous workflow.
The conversation weighs open-weight models against closed systems, arguing that privacy and customization matter while acknowledging the governance burden of giving agents more autonomy.
The closing question is larger than AI safety: if machines take over more manual work, how can people preserve privacy, agency, and a distinctly human future?
Weekly email
State of the Pod, every Monday
The week's most-discussed names and stories from podcasts, with the moments behind them. Free, five minutes, unsubscribe in one click.