
Jun 19, 2026 · 12 min
Sample efficiency defines the true trajectory of artificial intelligence
The data black hole at the center of AI
Understanding whether AI progress is driven by scaling or genuine algorithmic learning efficiency determines the ultimate limits of machine intelligence.
- 1Sample efficiency serves as a core metric for defining and measuring true algorithmic intelligence.
- 2Recent AI progress may rely more on scaling massive data and compute than genuine learning improvements.
- 3The reliance on massive data scaling raises critical questions about the future trajectory of AI development.
The brief
Recent breakthroughs in artificial intelligence have stunned the world, but they mask a fundamental question about how these systems actually learn. We must ask whether AI is truly getting smarter or just consuming massive amounts of data.
True intelligence can be defined by sample efficiency, which measures how effectively an algorithm learns from limited data. While humans can master concepts from a few examples, modern AI still requires vast oceans of information to achieve competency.
This raises a critical tension for the future of technology. If progress relies entirely on scaling up data and compute rather than improving algorithmic efficiency, we may soon hit a hard ceiling as high-quality training data runs out.
What was said on this episode
29 statements · 12 positive · 12 negative · 1 mixed · 4 neutral
AI training sample efficiency has not clearly improved much in recent years
“it's actually not clear that we've made that much progress in training sample efficiency over the last few years”
Listen at 0:07
AI progress has mainly come from better data and more data-development compute
“The main way that AIs have been getting better is from adding more and better data and scaling the compute required to develop that data in the first place.”
Listen at 0:18
Reinforcement learning requires models to have prior probability for correct solutions.
“For this process to work, the model must have at least some prior probability to anticipate the correct solution in the first place.”
Listen at 0:47
AI competence across fields requires vast human-expert trajectories for each skill.
“Which is why you need mind stretching amounts of human expert trajectories in every single field and skill that you want the model to eventually be competent in.”
Listen at 0:53
AI competence across skills requires vast amounts of expert human trajectories
“you need mind stretching amounts of human expert trajectories in every single field and skill that you want the model to eventually be competent in”
Listen at 0:54
Each AI skill requires at least hundreds of human experts producing examples, rubrics, and reasoning.
“Each skill corresponds to at least hundreds of human experts who are generating example completions, writing rubrics and explaining their chain of thought.”
Listen at 1:28
The AI data and reinforcement-learning-environment industry earns billions annually and may soon reach tens of billions.
“There's a reason that the data industry that is producing these expert labels and the RL environments in which these meticulously cataloged skills can congeal, is earning billions a year in revenue, soon to be deca. Billions.”
Listen at 1:36
The expert-label and RL-environment data industry earns billions annually
“the data industry that is producing these expert labels and the RL environments in which these meticulously cataloged skills can congeal, is earning billions a year in revenue”
Listen at 1:37
Data-driven progress makes it relatively easy for open models and laggards to approach frontier systems.
“I think the reason it is relatively easy for open source and previous laggards to catch up to within months of the frontier is that data is the real driver of progress.”
Listen at 2:29
Data is the primary driver of AI progress
“data is the real driver of progress”
Listen at 2:36
Public APIs make AI training data easier to distill than technical optimization methods.
“And data can be easily distilled from public APIs, whereas hyperparameters and training tricks and architectural optimizations cannot.”
Listen at 2:39
Frontier models train on tens to hundreds of trillions of tokens
“these frontier models are trained on somewhere between tens to hundreds of trillions of tokens”
Listen at 3:27
Current AI systems learn substantially less efficiently than humans.
“But the reason we can't do this is that our AIs learn much less efficiently than we do.”
Listen at 3:51
Current AI systems learn less efficiently than humans
“our AIs learn much less efficiently than we do”
Listen at 3:52
Millions of demonstration hours do not enable AI to perform complex open-ended tasks.
“And even with the millions of hours of demonstrations that we've collected, this is not enough to to allow them to perform complex open ended tasks.”
Listen at 3:56
Human driving requires far less training data than Waymo and Tesla models use
“that is still three to four orders of magnitude less data than Waymo and Tesla are using to train their self driving car models”
Listen at 4:13
The human genome is approximately three gigabytes, with one to two percent protein-coding.
“Our genome is only 3 gigabytes big and only 1 to 2% of it is protein coding.”
Listen at 4:46
Evolution supplied learning hyperparameters and loss functions, while individuals build their brain connectomes.
“I think the closer analogy is more that evolution found the right hyperparameters and the right loss functions and that within our lifetime we are still from scratch building up the connectome in our brain”
Listen at 4:59
Evolution supplied humans with effective hyperparameters and loss functions
“evolution found the right hyperparameters and the right loss functions”
Listen at 5:02
Pretrained AI still requires enormous data for each additional skill.
“But these AIs, even once they're pre trained, still require enormous amounts of data to learn the next marginal skill and the next marginal skill after that.”
Listen at 5:39
Blind and deaf people retain general intelligence without typical sensory input
“blind and deaf people who have been cut off from all the sensory information still have general intelligence”
Listen at 6:00
The human-AI data gap may exceed one millionfold
“even the million fold difference that we calculated earlier might be an understatement”
Listen at 6:25
Unlimited parameter growth would reduce required data by only tenfold at equal loss
“Even if you increase the number of parameters by infinity, that would only decrease by a factor of 10 the amount of data that you need in order to keep the same loss.”
Listen at 7:14
Humans are thousands to millions of times more sample-efficient than current AI models.
“Humans are somewhere between thousands to millions of times more sample efficient than these models.”
Listen at 7:23
Scaling current model size alone cannot close the human-AI sample-efficiency gap.
“So scaling the size of current models simply can't make up for that discrepancy.”
Listen at 7:28
Training on common tasks creates substantial economic value
“it does suggest that there's an enormous amount of value from bringing into distribution these kinds of common tasks”
Listen at 9:16
AI training can remain highly inefficient while still being profitable
“we can be ludicrously inefficient in training them up and still be wildly in the green”
Listen at 10:05
Demand for human software engineers will be higher in 2027
“I would be willing to bet that there's overall more demand for human software engineers in 2027 than there is right now”
Listen at 10:44
AI complements human software engineers and increases their demand
“largely due to the complementary input of AI”
Listen at 10:50
Statements are attributed to the speaker as said on the episode and reflect their view at the time, not PodLume's. They are not advice.
Weekly email
State of the Pod, every Monday
The week's most-discussed names and stories from podcasts, with the moments behind them. Free, five minutes, unsubscribe in one click.