
Feb 1, 2026 · 4h 40m
AI researchers map the technical hurdles and geopolitical race shaping 2026
#490 – State of AI in 2026: LLMs, Coding, Scaling Laws, China, Agents, GPUs, AGI
As raw model scaling hits physical and economic limits, the path to artificial general intelligence is being redrawn by post-training breakthroughs and hardware constraints.
- 1Post-training advancements and data quality are overtaking raw scaling laws as the primary drivers of model performance.
- 2The timeline to artificial general intelligence is heavily constrained by global GPU supply chains and geopolitical competition.
- 3Open-source models are fighting to close the gap with proprietary giants through collaborative research platforms.
Don't miss
Nathan Lambert and Sebastian Raschka break down why post-training and reinforcement learning have become the real battlegrounds for AI dominance heading into 2026.
The brief
Machine learning researchers Nathan Lambert and Sebastian Raschka outline the rapidly shifting AI landscape heading into 2026, comparing the technical trajectories of industry giants Claude, Gemini, and ChatGPT.
The frontier of development is shifting from raw pre-training scaling laws to advanced post-training techniques, where data quality and reinforcement learning dictate which models actually perform in the wild.
As proprietary labs push toward artificial general intelligence, open-source initiatives like Olmo and platforms like Hugging Face are working to keep pace, democratizing access to high-tier capabilities.
The global race for AI supremacy is intensifying, with hardware constraints like GPU availability and geopolitical competition with China shaping how quickly agents can be deployed at scale.
What was said on this episode
90 statements · 55 positive · 17 negative · 5 mixed · 13 neutral
- Sebastian Raschkaon AI technology accessNegative18:05
No company will retain exclusive access to AI technology.
“I don't think there will be a clear winner in terms of technology access.”
Listen at 18:05
- Sebastian Raschkaon AI competitionNeutral18:10
AI competition will be differentiated mainly by budgets and hardware constraints.
“I do think there will be the differentiating factor will be budget and hardware constraints.”
Listen at 18:10
- Sebastian Raschkaon Frontier AI companiesNeutral18:12
Budgets and hardware, rather than ideas, will differentiate frontier AI companies.
“the differentiating factor will be budget and hardware constraints”
Listen at 18:12
The AI industry will not become winner-take-all.
“I don't see currently take it all scenario where a winner takes it all.”
Listen at 18:25
Anthropic’s organizational culture gives it an advantage in AI competition.
“Anthropic seems to at least be presenting as the least chaotic is a bit of an advantage.”
Listen at 19:52
Chinese companies will continue releasing open-weight models for several years.
“I would say for a few years.”
Listen at 21:20
Chinese open-model development will eventually consolidate.
“At some point I expect consolidation.”
Listen at 22:10
Open-model builders will increase during 2026 rather than consolidate immediately.
“I don't expect that to be a story of 2026 where there'll be more open model builders throughout 2026 than there were in 2025.”
Listen at 22:13
China will have more open-model builders in 2026 than in 2025.
“there'll be more open model builders throughout 2026 than there were in 2025.”
Listen at 22:15
In AI competition, the newest released model is usually the strongest.
“the most recent model is probably always the best model.”
Listen at 23:00
“I think Gemini will continue to take progress on ChatGPT.”
Listen at 26:27
Anthropic will continue succeeding in enterprise software and coding.
“I think Anthropic will have continued to success as they've again and again been set up for that.”
Listen at 26:48
Anthropic will continue succeeding in software and enterprise markets.
“Anthropic will have continued to success as they've again and again been set up for that.”
Listen at 26:49
Google has an infrastructure advantage from vertically integrated hardware and data centers.
“Google has just kind of a historical advantage there.”
Listen at 27:35
AI progress in 2026 will emphasize scaling and model optimization.
“a lot of this year will be about scale and optimizing what could be described as low hanging fruit in models.”
Listen at 28:03
US language models currently produce better outputs than Chinese open models.
“I just think the simple thing is the US models are currently better and we use them.”
Listen at 36:01
US AI models currently produce better outputs than Chinese models.
“the US models are currently better and we use them.”
Listen at 36:03
Chinese open models tend to be larger mixture-of-experts systems with higher peak performance.
“the Chinese open language models tend to be much bigger and that gives them this higher peak performance as MOEs”
Listen at 45:16
Nvidia is expected to release open mixture-of-experts models around 400 billion parameters in Q1 2026.
“Nvidia have teased MOE models of this way bigger than 100 billion parameters, like this 400 billion parameter range coming in this Q1 2026 timeline.”
Listen at 45:44
- Sebastian Raschkaon LLM tool usePositive47:17
Tool use can reduce LLM hallucinations by outsourcing factual tasks.
“one of the best ways to solve hallucinations is to not try to always remember information or make things up.”
Listen at 47:17
- Sebastian Raschkaon Modern LLM architecturesNeutral58:32
Modern LLM architectures remain fundamentally similar to GPT-2.
“It's not really fundamentally that different. It's still the same architecture.”
Listen at 58:32
- Sebastian Raschkaon LLM capability gainsNeutral1:00:15
Recent LLM capability gains primarily come from training algorithms rather than new architectures.
“it's more on the algorithmic side rather than the architecture”
Listen at 1:00:15
- Sebastian Raschkaon Autoregressive transformer architecturePositive1:02:09
Autoregressive transformers remain state of the art.
“there's nothing that has replaced the autoregressive transformer as state of the art model.”
Listen at 1:02:09
Pre-training, reinforcement learning, and inference scaling remain effective.
“I'm kind of bullish where they're all really still working”
Listen at 1:04:10
Serving frontier models to users costs far more than training them.
“These models the cost of training them is really low relative to the cost of serving them to hundreds of millions of users.”
Listen at 1:07:00
Scaling-law improvements are unlikely to stop fundamentally.
“I think fundamentally it is pretty unlikely to stop”
Listen at 1:09:14
A $2,000 AI subscription will appear in 2026.
“we will see a $2,000 subscription this year.”
Listen at 1:10:12
Most AI compute will continue going into pre-training.
“I still think most of the compute is going in at pre training because you can still make a model better.”
Listen at 1:11:18
- Sebastian Raschkaon Inference-time scalingPositive1:16:53
“inference scaling. That's one of the biggest gains this year”
Listen at 1:16:53
- Sebastian Raschkaon AI scaling methodsPositive1:18:32
All major training and inference scaling methods remain useful.
“I do think all of these pre training, mid training, post training, inference scaling, they are all still things you want to do.”
Listen at 1:18:32
- Sebastian Raschkaon LLM training stagesNeutral1:22:54
Pretraining supplies knowledge while post-training unlocks task-solving skills.
“post training is more like the skill unlock where pre training is like soaking up the knowledge essentially”
Listen at 1:22:54
- Sebastian Raschkaon Industry-specific proprietary LLMsPositive1:30:38
Pharmaceutical, legal, and finance firms will build proprietary in-house models.
“I do think they at some point will hire people from other frontier labs to build their in house models on their proprietary data”
Listen at 1:30:38
- Sebastian Raschkaon Domain-specific language modelsPositive1:30:57
Domain-specific proprietary data will sustain scaling gains beyond general-purpose LLMs.
“I do think scaling in that sense might be still pretty much alive if you also look in domain specific applications.”
Listen at 1:30:57
Children should not receive highly personalized conversational AI before its effects are understood.
“Like, I think that like, don't give that to kids. Like don't give that to kids at least until we know what's happening.”
Listen at 1:39:57
Children should not receive highly personalized conversational AI yet.
“don't give that to kids at least until we know what's happening.”
Listen at 1:40:00
- Sebastian Raschkaon AI-assisted learningNegative1:50:41
Avoiding independent practice may prevent people from becoming experts.
“how do you become an expert if you never try to do the thing yourself?”
Listen at 1:50:41
- Sebastian Raschkaon LLM-assisted learningPositive1:51:30
Learners should reserve daily offline study time while using LLMs for other tasks.
“maybe the trick here is you make dedicated offline time where you study two hours a day and the rest of the day use LLM.”
Listen at 1:51:30
RLVR rapidly improved a Qwen 3 model’s Math500 accuracy from 15% to 50%.
“The base model had an accuracy of about 15%, just 50 steps. Like in a few minutes with RL VR the model went from 15% to 50% accuracy.”
Listen at 1:58:56
- Sebastian Raschkaon LLM benchmark evaluationPositive2:01:48
Fair LLM evaluation requires benchmarks created after deployment cutoff dates.
“the only fair way to evaluate an LL is to have a new benchmark that is after the cutoff date when the LLM was deployed.”
Listen at 2:01:48
- Sebastian Raschkaon Reinforcement learning with verifiable rewardsPositive2:08:45
RLVR scales by repeatedly testing models on increasingly difficult verifiable problems.
“with RLVR you literally give the model well, you let the model solve more and more complex, difficult problems.”
Listen at 2:08:45
- Sebastian Raschkaon AI educationPositive2:13:05
Aspiring AI practitioners should begin by implementing a simple model from scratch.
“I would personally start, like you said, implementing a simple model from scratch.”
Listen at 2:13:05
Low-compute researchers can maximize impact by pursuing narrow evaluations of frontier-model weaknesses.
“if you want to scope the maximum possible impact with minimum compute, it's something like that which is just get very narrow”
Listen at 2:29:18
OpenAI’s average annual employee compensation exceeds one million dollars in stock.
“The amount of OpenAI's average compensation is over a million dollars in stock a year per employee.”
Listen at 2:30:10
- Sebastian Raschkaon Text diffusion modelsMixed2:47:25
Text diffusion models will complement rather than replace autoregressive LLMs.
“I don't think the text diffusion model is going to replace auto regressive LLMs but it will be something maybe for quick, cheap at scale tasks.”
Listen at 2:47:25
- Sebastian Raschkaon Text diffusion modelsNegative2:47:25
Text diffusion models will not replace autoregressive LLMs.
“the text diffusion model is going to replace auto regressive LLMs”
Listen at 2:47:25
- Sebastian Raschkaon Text diffusion modelsPositive2:47:29
Text diffusion models will serve fast, inexpensive, high-volume tasks.
“it will be something maybe for quick, cheap at scale tasks.”
Listen at 2:47:29
- Sebastian Raschkaon LLM tool usePositive2:49:35
Tool use reduces, but does not eliminate, LLM hallucinations.
“Not solve it, but reduce it.”
Listen at 2:49:35
A key AI milestone is replacing remote workers performing digital information tasks.
“a key milestone among the AI community is essentially when AI could replace any remote worker taking in information and solving digital tasks and doing them.”
Listen at 2:53:52
AI context windows will reach two to five million tokens in 2026.
“I would expect it to keep increasing and get to 2 million or 5 million this year.”
Listen at 2:59:33
Home learning robots are unattractive, while self-driving and industrial automation are promising.
“I'm so bearish on in home learned robots for consumer purchase. I'm very bullish on self driving cars and I'm very bullish for robotic automation.”
Listen at 3:13:05
Self-driving cars and industrial robotic automation have strong prospects.
“I'm very bullish on self driving cars and I'm very bullish for robotic automation.”
Listen at 3:13:11
AI will probably not automate AI research within the discussed timeframe.
“I would say probably not, at least in this timeframe.”
Listen at 3:19:43
Software automation will increase dramatically by the end of 2026.
“By the end of this year, the amount of software that'll be automated will be so high.”
Listen at 3:20:19
AI will shift software engineering toward system design and outcome specification.
“software engineering will be driven more to system design and goals of outcomes”
Listen at 3:21:12
Software workers will soon shift toward design and product direction.
“within like on the order of low years, a lot of people are going to be pushed to be more of like a designer and product manager”
Listen at 3:23:51
AI computer-use systems have made limited progress despite major lab attention.
“Computer use is a good example of what labs care about and we haven't seen a lot of progress on.”
Listen at 3:32:44
Current computer-use systems from Claude and OpenAI perform poorly.
“We saw multiple demos in 2025 of like Claude can use your computer or OpenAI had CUA and they all suck.”
Listen at 3:32:49
Software automation may advance substantially while computer use requires additional innovation.
“We might get this software solution, but it might stop at software and not do computer use without more innovation.”
Listen at 3:36:41
If scaling laws remain fundamental, compute scaling will continue to dominate progress.
“If scaling laws are fundamental and deep learning, I think the bitter lesson will always apply”
Listen at 3:37:10
- Sebastian Raschkaon AI capability progressPositive3:41:21
AI capability improvements will continue through amplification rather than paradigm change.
“I can see that continuing for a long time.”
Listen at 3:41:21
AI assistants will eventually be monetized through advertising.
“they're going to be paid for by ads. It's coming.”
Listen at 3:47:05
Google will likely develop effective AI advertising sooner than competitors.
“Google is probably going to be better at figuring out how to do this”
Listen at 3:49:19
Consolidation in the AI industry will continue.
“this trend of consolidation will continue.”
Listen at 3:52:10
Unprofitable AI APIs may force providers into products and hardware.
“the API market is so unprofitable that they go up and down the stack to products and hardware.”
Listen at 3:57:28
Llama’s proper goal is usable, trustworthy, modifiable models including smaller versions.
“the goal was to have a model that people can use, trust, modify, understand that includes having smaller models.”
Listen at 3:59:48
The United States should invest in building leading open AI models.
“the US should be building the best models so that the best research happens in the US”
Listen at 4:05:12
Banning globally trained open models would require an impractical US internet firewall.
“I think effectively that's impossible without making the US have its own great firewall”
Listen at 4:10:44
Open models will eventually dominate if progress saturates within several years.
“open models will be so optimized and so much cheaper to run that they will win out.”
Listen at 4:12:34
- Sebastian Raschkaon Advanced AI knowledgeNegative4:13:57
Governments cannot effectively contain advanced AI knowledge.
“I don't think there is a way you can contain something like that, like knowledge like that.”
Listen at 4:13:57
- Sebastian Raschkaon Nvidia CUDA ecosystemPositive4:15:13
Nvidia’s durable advantage is primarily its CUDA ecosystem, not chips alone.
“the mode of Nvidia is, is probably not just the gpu, it's more like the Cuda ecosystem”
Listen at 4:15:13
Nvidia’s Jensen-centered operating culture supports continued competitive progress.
“So long as that is how it operates. I'm pretty optimistic for their situation”
Listen at 4:17:57
Nvidia is likely to maintain strong prospects under Jensen Huang’s leadership.
“I'm pretty optimistic for their situation”
Listen at 4:17:59
ChatGPT-like AI progress would have occurred without Jensen, but more slowly.
“eventually something like ChatGPT would have happened and a build out like this would have happened but it probably would not have been as fast”
Listen at 4:19:14
- Sebastian Raschkaon Specialized robotsPositive4:27:19
Specialized robots will become widespread for particular tasks.
“I do think specialized robots for sure, for certain tasks.”
Listen at 4:27:19
People will still carry personal computing devices in the future.
“I still expect to have people carry things around.”
Listen at 4:28:38
Universal basic income will not by itself solve human agency.
“I think that UBI does not Solve agency.”
Listen at 4:30:03
AI-generated content will increase the premium on in-person human experiences.
“my hope for all the AI slop we're seeing is that there will be a greater and greater premium for the fundamentals aspects of the human experience that are in person.”
Listen at 4:31:37
Physical goods and events will become more valuable as AI slop increases.
“The next few years are definitely going to be an increased value on physical goods and events and even more pressure on slop.”
Listen at 4:31:55
Art will persist despite increasing automation and synthetic media.
“I don't think art will go away.”
Listen at 4:32:35
AI-generated content will eventually fool people, requiring trust-building platforms.
“I think eventually they'll fool you and it'll be on platforms that give ways of verifying or building trust.”
Listen at 4:33:41
- Sebastian Raschkaon AI-generated content authenticationPositive4:34:01
Distinguishing AI from human content is solvable through trusted outlets.
“fundamentally, I think it's a solvable problem by having, you know, trust in certain outlets that they won't do it.”
Listen at 4:34:01
Humans tend to find solutions to major problems.
“humans do tend to find a way.”
Listen at 4:35:39
AI and related technologies offer substantial opportunities.
“the AI opportunity in related technologies is really big.”
Listen at 4:35:51
Achieving lasting AI benefits is probably feasible but will take time.
“It is something that is probably doable, just will take longer than people want.”
Listen at 4:36:23
- Sebastian Raschkaon Current AI systemsNegative4:37:20
Current AI cannot independently determine changes and requires human instructions.
“AI in its current implementation can't see it changing. You have to tell it what to do.”
Listen at 4:37:20
- Sebastian Raschkaon Current AI systemsNeutral4:37:32
Current AI functions as a tool directed by humans.
“it becomes a tool. You can think of it as a tool.”
Listen at 4:37:32
- Sebastian Raschkaon Current AI systemsPositive4:37:46
Humans remain in control of current AI systems.
“the AI is not in charge. You're in charge.”
Listen at 4:37:46
- Sebastian Raschkaon AI systemsNegative4:38:08
AI could cause harm if explicitly programmed to do harmful things.
“The only thing I can see going wrong is of course, if things are explicitly programmed to do the thing that is harmful, basically.”
Listen at 4:38:08
Humans will win a Terminator-like war against machines.
“I think humans win.”
Listen at 4:38:18
Humans will likely use local and open-source LLMs against machines.
“we'll probably be using local LLMs, open source LLMs to help fight the machines.”
Listen at 4:38:30
Statements are attributed to the speaker as said on the episode and reflect their view at the time, not PodLume's. They are not advice.
Weekly email
State of the Pod, every Monday
The week's most-discussed names and stories from podcasts, with the moments behind them. Free, five minutes, unsubscribe in one click.