
Nov 17, 2026 · 18 min
Listen from 15:41
Listen at 15:41
AI research may automate itself—but judgment remains the bottleneck
Can AI Learn to Improve Itself? | Recursive Self-Improvement
Recursive self-improvement could accelerate AI capabilities and alignment, but only if systems can evaluate ideas, manage compute, and reflect chosen human values.
- 1Current AI research assistance accelerates human-led work but remains far from fully autonomous discovery and feedback loops.
- 2Open-ended research needs harder, durable evaluations, while compute constraints limit how quickly systems can test and improve ideas.
- 3More autonomous systems intensify alignment and governance questions, including whose values models should represent and how those choices are made.
Don't miss
The guests’ most consequential turn links recursive self-improvement to the political question of whose values increasingly autonomous AI should reflect.
The brief
Sarah Wang, Mark Chen, and John Schulman examine recursive self-improvement: the possibility that AI systems could automate and accelerate the research needed to improve AI itself.
Today’s systems can assist with ideas and experiments, but the guests place current progress much earlier on the RSI spectrum than on the AGI spectrum.
The central bottleneck is research judgment: models must recognize ideas that are important, generalizable, and composable, not merely generate plausible experiments.
Automated research also needs reliable evaluation and abundant compute; without harder benchmarks and affordable testing, there is no dependable hill-climbing loop.
The discussion connects RSI directly to alignment, asking whether stronger systems can help align other models and who should decide the values those systems reflect.
The working theory is cautiously optimistic: AI may help solve alignment, but human direction and judgment are likely to remain important for the foreseeable future.
Weekly email
State of the Pod, every Monday
The week's most-discussed names and stories from podcasts, with the moments behind them. Free, five minutes, unsubscribe in one click.