Feb 1, 2026 · 4h 40m
AI researchers map the technical hurdles and geopolitical race shaping 2026
#490 – State of AI in 2026: LLMs, Coding, Scaling Laws, China, Agents, GPUs, AGI
As raw model scaling hits physical and economic limits, the path to artificial general intelligence is being redrawn by post-training breakthroughs and hardware constraints.
- 1Post-training advancements and data quality are overtaking raw scaling laws as the primary drivers of model performance.
- 2The timeline to artificial general intelligence is heavily constrained by global GPU supply chains and geopolitical competition.
- 3Open-source models are fighting to close the gap with proprietary giants through collaborative research platforms.
Don't miss
Nathan Lambert and Sebastian Raschka break down why post-training and reinforcement learning have become the real battlegrounds for AI dominance heading into 2026.
The brief
Machine learning researchers Nathan Lambert and Sebastian Raschka outline the rapidly shifting AI landscape heading into 2026, comparing the technical trajectories of industry giants Claude, Gemini, and ChatGPT.
The frontier of development is shifting from raw pre-training scaling laws to advanced post-training techniques, where data quality and reinforcement learning dictate which models actually perform in the wild.
As proprietary labs push toward artificial general intelligence, open-source initiatives like Olmo and platforms like Hugging Face are working to keep pace, democratizing access to high-tier capabilities.
The global race for AI supremacy is intensifying, with hardware constraints like GPU availability and geopolitical competition with China shaping how quickly agents can be deployed at scale.
What was said on this episode
50 statements · 32 positive · 8 negative · 5 mixed · 5 neutral
No company will retain exclusive access to AI technology.
“I don't think there will be a clear winner in terms of technology access.”
Listen at 18:05
Budgets and hardware, rather than ideas, will differentiate frontier AI companies.
“the differentiating factor will be budget and hardware constraints”
Listen at 18:12
The AI industry will not become winner-take-all.
“I don't see currently take it all scenario where a winner takes it all.”
Listen at 18:25
Anthropic’s relatively orderly organizational culture provides a competitive advantage.
“Anthropic seems to at least be presenting as the least chaotic is a bit of an advantage.”
Listen at 19:52
Chinese companies will continue releasing open-weight models for several years.
“I would say for a few years.”
Listen at 21:20
Open-model builders will increase during 2026 rather than consolidate immediately.
“I don't expect that to be a story of 2026 where there'll be more open model builders throughout 2026 than there were in 2025.”
Listen at 22:13
In AI competition, the newest released model is usually the strongest.
“the most recent model is probably always the best model.”
Listen at 23:00
Gemini will continue gaining ground against ChatGPT during 2026.
“I think Gemini will continue to take progress on ChatGPT.”
Listen at 26:27
Anthropic will continue succeeding in enterprise software and coding.
“I think Anthropic will have continued to success as they've again and again been set up for that.”
Listen at 26:48
Google has an infrastructure advantage from vertically integrated hardware and data centers.
“Google has just kind of a historical advantage there.”
Listen at 27:35
US language models currently produce better outputs than Chinese open models.
“I just think the simple thing is the US models are currently better and we use them.”
Listen at 36:01
Chinese open models tend to be larger mixture-of-experts systems with higher peak performance.
“the Chinese open language models tend to be much bigger and that gives them this higher peak performance as MOEs”
Listen at 45:16
Nvidia is expected to release open mixture-of-experts models around 400 billion parameters in Q1 2026.
“Nvidia have teased MOE models of this way bigger than 100 billion parameters, like this 400 billion parameter range coming in this Q1 2026 timeline.”
Listen at 45:44
Tool use can reduce LLM hallucinations by replacing memorization with external verification.
“one of the best ways to solve hallucinations is to not try to always remember information or make things up”
Listen at 47:17
Modern LLM architectures remain fundamentally similar to GPT-2.
“It's not really fundamentally that different. It's still the same architecture.”
Listen at 58:32
Recent LLM capability gains primarily come from training algorithms rather than new architectures.
“it's more on the algorithmic side rather than the architecture”
Listen at 1:00:15
Autoregressive transformers remain state of the art despite emerging alternatives.
“there's nothing that has replaced the autoregressive transformer as state of the art model”
Listen at 1:02:09
Serving frontier models to users costs far more than training them.
“These models the cost of training them is really low relative to the cost of serving them to hundreds of millions of users.”
Listen at 1:07:00
Pretraining scaling is unlikely to stop improving model performance.
“I think fundamentally it is pretty unlikely to stop”
Listen at 1:09:14
A $2,000-per-month AI subscription will appear during 2026.
“we will see a $2,000 subscription this year.”
Listen at 1:10:12
All major training and inference scaling methods remain useful.
“I do think all of these pre training, mid training, post training, inference scaling, they are all still things you want to do.”
Listen at 1:18:32
Pretraining supplies knowledge while post-training unlocks task-solving skills.
“post training is more like the skill unlock where pre training is like soaking up the knowledge essentially”
Listen at 1:22:54
Domain-specific proprietary data will sustain scaling gains beyond general-purpose LLMs.
“I do think scaling in that sense might be still pretty much alive if you also look in domain specific applications.”
Listen at 1:30:57
Children should not receive highly personalized conversational AI before its effects are understood.
“Like, I think that like, don't give that to kids. Like don't give that to kids at least until we know what's happening.”
Listen at 1:39:57
Learners should reserve daily offline study time while using LLMs for other tasks.
“maybe the trick here is you make dedicated offline time where you study two hours a day and the rest of the day use LLM.”
Listen at 1:51:30
RLVR rapidly improved a Qwen 3 model’s Math500 accuracy from 15% to 50%.
“The base model had an accuracy of about 15%, just 50 steps. Like in a few minutes with RL VR the model went from 15% to 50% accuracy.”
Listen at 1:58:56
Fair LLM evaluation requires benchmarks created after deployment cutoff dates.
“the only fair way to evaluate an LL is to have a new benchmark that is after the cutoff date when the LLM was deployed.”
Listen at 2:01:48
RLVR scales by repeatedly testing models on increasingly difficult verifiable problems.
“with RLVR you literally give the model well, you let the model solve more and more complex, difficult problems.”
Listen at 2:08:45
Aspiring AI practitioners should begin by implementing a simple model from scratch.
“I would personally start, like you said, implementing a simple model from scratch.”
Listen at 2:13:05
Low-compute researchers can maximize impact by pursuing narrow evaluations of frontier-model weaknesses.
“if you want to scope the maximum possible impact with minimum compute, it's something like that which is just get very narrow”
Listen at 2:29:18
OpenAI’s average annual employee compensation exceeds one million dollars in stock.
“The amount of OpenAI's average compensation is over a million dollars in stock a year per employee.”
Listen at 2:30:10
Text diffusion models will complement rather than replace autoregressive LLMs.
“I don't think the text diffusion model is going to replace auto regressive LLMs but it will be something maybe for quick, cheap at scale tasks.”
Listen at 2:47:25
Tool use reduces, but does not eliminate, LLM hallucinations.
“Not solve it, but reduce it.”
Listen at 2:49:35
A key AI milestone is replacing remote workers performing digital information tasks.
“a key milestone among the AI community is essentially when AI could replace any remote worker taking in information and solving digital tasks and doing them.”
Listen at 2:53:52
LLM context windows may reach 2–5 million tokens in 2026, but not 100 million.
“I would expect it to keep increasing and get to 2 million or 5 million this year. But I don't expect it to go to 100 million.”
Listen at 2:59:33
Home learning robots are unattractive, while self-driving and industrial automation are promising.
“I'm so bearish on in home learned robots for consumer purchase. I'm very bullish on self driving cars and I'm very bullish for robotic automation.”
Listen at 3:13:05
AI is unlikely to automate AI research within the timeframe discussed.
“I would say probably not, at least in this timeframe.”
Listen at 3:19:43
Software automation will increase dramatically by the end of 2026.
“By the end of this year, the amount of software that'll be automated will be so high.”
Listen at 3:20:19
AI will shift software engineering toward system design and outcome specification.
“software engineering will be driven more to system design and goals of outcomes”
Listen at 3:21:12
Current computer-use systems from Claude and OpenAI perform poorly.
“We saw multiple demos in 2025 of like Claude can use your computer or OpenAI had CUA and they all suck.”
Listen at 3:32:49
Software automation may advance substantially while computer use requires additional innovation.
“We might get this software solution, but it might stop at software and not do computer use without more innovation.”
Listen at 3:36:41
If scaling laws remain fundamental, increasing compute will continue driving deep-learning progress.
“If scaling laws are fundamental and deep learning, I think the bitter lesson will always apply, which is compute will become more abundant.”
Listen at 3:37:10
AI capability improvements will continue through amplification rather than a paradigm shift.
“I can see that continuing for a long time.”
Listen at 3:41:21
The United States should invest in building leading open AI models.
“the US should be building the best models so that the best research happens in the US”
Listen at 4:05:12
Banning globally trained open models would require an impractical US internet firewall.
“I think effectively that's impossible without making the US have its own great firewall”
Listen at 4:10:44
If progress saturates soon, optimized open models will win because they are cheaper to run.
“open models will be so optimized and so much cheaper to run that they will win out.”
Listen at 4:12:34
Nvidia’s main competitive moat is the CUDA ecosystem rather than GPU hardware alone.
“the mode of Nvidia is, is probably not just the gpu, it's more like the Cuda ecosystem”
Listen at 4:15:13
Nvidia’s Jensen-centered operating culture supports continued competitive progress.
“So long as that is how it operates. I'm pretty optimistic for their situation”
Listen at 4:17:57
Universal basic income will not by itself solve human agency.
“I think that UBI does not Solve agency.”
Listen at 4:30:03
AI-generated slop will increase demand for physical goods and in-person events.
“The next few years are definitely going to be an increased value on physical goods and events and even more pressure on slop.”
Listen at 4:31:55
Statements are attributed to the speaker as said on the episode and reflect their view at the time, not PodLume's. They are not advice.