Lex Fridman Podcast
Lex Fridman Podcast

Feb 1, 2026 · 4h 40m

AI researchers map the technical hurdles and geopolitical race shaping 2026

#490 – State of AI in 2026: LLMs, Coding, Scaling Laws, China, Agents, GPUs, AGI

As raw model scaling hits physical and economic limits, the path to artificial general intelligence is being redrawn by post-training breakthroughs and hardware constraints.

3 key takeaways
  1. 1Post-training advancements and data quality are overtaking raw scaling laws as the primary drivers of model performance.
  2. 2The timeline to artificial general intelligence is heavily constrained by global GPU supply chains and geopolitical competition.
  3. 3Open-source models are fighting to close the gap with proprietary giants through collaborative research platforms.

Don't miss

Nathan Lambert and Sebastian Raschka break down why post-training and reinforcement learning have become the real battlegrounds for AI dominance heading into 2026.

The brief

Machine learning researchers Nathan Lambert and Sebastian Raschka outline the rapidly shifting AI landscape heading into 2026, comparing the technical trajectories of industry giants Claude, Gemini, and ChatGPT.

The frontier of development is shifting from raw pre-training scaling laws to advanced post-training techniques, where data quality and reinforcement learning dictate which models actually perform in the wild.

As proprietary labs push toward artificial general intelligence, open-source initiatives like Olmo and platforms like Hugging Face are working to keep pace, democratizing access to high-tier capabilities.

The global race for AI supremacy is intensifying, with hardware constraints like GPU availability and geopolitical competition with China shaping how quickly agents can be deployed at scale.

What was said on this episode

90 statements · 55 positive · 17 negative · 5 mixed · 13 neutral

  1. Sebastian Raschkaon AI technology accessNegative18:05

    No company will retain exclusive access to AI technology.

    “I don't think there will be a clear winner in terms of technology access.”

    Listen at 18:05

  2. Sebastian Raschkaon AI competitionNeutral18:10

    AI competition will be differentiated mainly by budgets and hardware constraints.

    “I do think there will be the differentiating factor will be budget and hardware constraints.”

    Listen at 18:10

  3. Sebastian Raschkaon Frontier AI companiesNeutral18:12

    Budgets and hardware, rather than ideas, will differentiate frontier AI companies.

    “the differentiating factor will be budget and hardware constraints”

    Listen at 18:12

  4. Sebastian Raschkaon AI industryNegative18:25

    The AI industry will not become winner-take-all.

    “I don't see currently take it all scenario where a winner takes it all.”

    Listen at 18:25

  5. Nathan Lamberton AnthropicPositive19:52

    Anthropic’s organizational culture gives it an advantage in AI competition.

    “Anthropic seems to at least be presenting as the least chaotic is a bit of an advantage.”

    Listen at 19:52

  6. Chinese companies will continue releasing open-weight models for several years.

    “I would say for a few years.”

    Listen at 21:20

  7. Nathan Lamberton Chinese open-model companiesNeutral22:10

    Chinese open-model development will eventually consolidate.

    “At some point I expect consolidation.”

    Listen at 22:10

  8. Nathan Lamberton Open-model buildersPositive22:13

    Open-model builders will increase during 2026 rather than consolidate immediately.

    “I don't expect that to be a story of 2026 where there'll be more open model builders throughout 2026 than there were in 2025.”

    Listen at 22:13

  9. Nathan Lamberton Chinese open-model buildersPositive22:15

    China will have more open-model builders in 2026 than in 2025.

    “there'll be more open model builders throughout 2026 than there were in 2025.”

    Listen at 22:15

  10. Sebastian Raschkaon Frontier modelPositive23:00

    In AI competition, the newest released model is usually the strongest.

    “the most recent model is probably always the best model.”

    Listen at 23:00

  11. Nathan Lamberton Gemini versus ChatGPTPositive26:27
    “I think Gemini will continue to take progress on ChatGPT.”

    Listen at 26:27

  12. Nathan Lamberton AnthropicPositive26:48

    Anthropic will continue succeeding in enterprise software and coding.

    “I think Anthropic will have continued to success as they've again and again been set up for that.”

    Listen at 26:48

  13. Nathan Lamberton AnthropicPositive26:49

    Anthropic will continue succeeding in software and enterprise markets.

    “Anthropic will have continued to success as they've again and again been set up for that.”

    Listen at 26:49

  14. Nathan Lamberton GooglePositive27:35

    Google has an infrastructure advantage from vertically integrated hardware and data centers.

    “Google has just kind of a historical advantage there.”

    Listen at 27:35

  15. Nathan Lamberton AI modelsPositive28:03

    AI progress in 2026 will emphasize scaling and model optimization.

    “a lot of this year will be about scale and optimizing what could be described as low hanging fruit in models.”

    Listen at 28:03

  16. Nathan Lamberton US and Chinese language modelsPositive36:01

    US language models currently produce better outputs than Chinese open models.

    “I just think the simple thing is the US models are currently better and we use them.”

    Listen at 36:01

  17. Nathan Lamberton US AI models versus Chinese AI modelsPositive36:03

    US AI models currently produce better outputs than Chinese models.

    “the US models are currently better and we use them.”

    Listen at 36:03

  18. Chinese open models tend to be larger mixture-of-experts systems with higher peak performance.

    “the Chinese open language models tend to be much bigger and that gives them this higher peak performance as MOEs”

    Listen at 45:16

  19. Nathan Lamberton Nvidia Nemotron modelsPositive45:44

    Nvidia is expected to release open mixture-of-experts models around 400 billion parameters in Q1 2026.

    “Nvidia have teased MOE models of this way bigger than 100 billion parameters, like this 400 billion parameter range coming in this Q1 2026 timeline.”

    Listen at 45:44

  20. Sebastian Raschkaon LLM tool usePositive47:17

    Tool use can reduce LLM hallucinations by outsourcing factual tasks.

    “one of the best ways to solve hallucinations is to not try to always remember information or make things up.”

    Listen at 47:17

  21. Sebastian Raschkaon Modern LLM architecturesNeutral58:32

    Modern LLM architectures remain fundamentally similar to GPT-2.

    “It's not really fundamentally that different. It's still the same architecture.”

    Listen at 58:32

  22. Sebastian Raschkaon LLM capability gainsNeutral1:00:15

    Recent LLM capability gains primarily come from training algorithms rather than new architectures.

    “it's more on the algorithmic side rather than the architecture”

    Listen at 1:00:15

  23. Sebastian Raschkaon Autoregressive transformer architecturePositive1:02:09

    Autoregressive transformers remain state of the art.

    “there's nothing that has replaced the autoregressive transformer as state of the art model.”

    Listen at 1:02:09

  24. Nathan Lamberton AI scaling methodsPositive1:04:10

    Pre-training, reinforcement learning, and inference scaling remain effective.

    “I'm kind of bullish where they're all really still working”

    Listen at 1:04:10

  25. Serving frontier models to users costs far more than training them.

    “These models the cost of training them is really low relative to the cost of serving them to hundreds of millions of users.”

    Listen at 1:07:00

  26. Nathan Lamberton Scaling lawsPositive1:09:14

    Scaling-law improvements are unlikely to stop fundamentally.

    “I think fundamentally it is pretty unlikely to stop”

    Listen at 1:09:14

  27. Nathan Lamberton AI subscriptionPositive1:10:12

    A $2,000 AI subscription will appear in 2026.

    “we will see a $2,000 subscription this year.”

    Listen at 1:10:12

  28. Nathan Lamberton AI compute allocationNeutral1:11:18

    Most AI compute will continue going into pre-training.

    “I still think most of the compute is going in at pre training because you can still make a model better.”

    Listen at 1:11:18

  29. Sebastian Raschkaon Inference-time scalingPositive1:16:53
    “inference scaling. That's one of the biggest gains this year”

    Listen at 1:16:53

  30. Sebastian Raschkaon AI scaling methodsPositive1:18:32

    All major training and inference scaling methods remain useful.

    “I do think all of these pre training, mid training, post training, inference scaling, they are all still things you want to do.”

    Listen at 1:18:32

  31. Sebastian Raschkaon LLM training stagesNeutral1:22:54

    Pretraining supplies knowledge while post-training unlocks task-solving skills.

    “post training is more like the skill unlock where pre training is like soaking up the knowledge essentially”

    Listen at 1:22:54

  32. Sebastian Raschkaon Industry-specific proprietary LLMsPositive1:30:38

    Pharmaceutical, legal, and finance firms will build proprietary in-house models.

    “I do think they at some point will hire people from other frontier labs to build their in house models on their proprietary data”

    Listen at 1:30:38

  33. Sebastian Raschkaon Domain-specific language modelsPositive1:30:57

    Domain-specific proprietary data will sustain scaling gains beyond general-purpose LLMs.

    “I do think scaling in that sense might be still pretty much alive if you also look in domain specific applications.”

    Listen at 1:30:57

  34. Nathan Lamberton Personalized conversational AI for childrenNegative1:39:57

    Children should not receive highly personalized conversational AI before its effects are understood.

    “Like, I think that like, don't give that to kids. Like don't give that to kids at least until we know what's happening.”

    Listen at 1:39:57

  35. Nathan Lamberton Personalized conversational AI for childrenNegative1:40:00

    Children should not receive highly personalized conversational AI yet.

    “don't give that to kids at least until we know what's happening.”

    Listen at 1:40:00

  36. Sebastian Raschkaon AI-assisted learningNegative1:50:41

    Avoiding independent practice may prevent people from becoming experts.

    “how do you become an expert if you never try to do the thing yourself?”

    Listen at 1:50:41

  37. Sebastian Raschkaon LLM-assisted learningPositive1:51:30

    Learners should reserve daily offline study time while using LLMs for other tasks.

    “maybe the trick here is you make dedicated offline time where you study two hours a day and the rest of the day use LLM.”

    Listen at 1:51:30

  38. Sebastian Raschkaon Qwen 3Positive1:58:56

    RLVR rapidly improved a Qwen 3 model’s Math500 accuracy from 15% to 50%.

    “The base model had an accuracy of about 15%, just 50 steps. Like in a few minutes with RL VR the model went from 15% to 50% accuracy.”

    Listen at 1:58:56

  39. Sebastian Raschkaon LLM benchmark evaluationPositive2:01:48

    Fair LLM evaluation requires benchmarks created after deployment cutoff dates.

    “the only fair way to evaluate an LL is to have a new benchmark that is after the cutoff date when the LLM was deployed.”

    Listen at 2:01:48

  40. Sebastian Raschkaon Reinforcement learning with verifiable rewardsPositive2:08:45

    RLVR scales by repeatedly testing models on increasingly difficult verifiable problems.

    “with RLVR you literally give the model well, you let the model solve more and more complex, difficult problems.”

    Listen at 2:08:45

  41. Sebastian Raschkaon AI educationPositive2:13:05

    Aspiring AI practitioners should begin by implementing a simple model from scratch.

    “I would personally start, like you said, implementing a simple model from scratch.”

    Listen at 2:13:05

  42. Nathan Lamberton AI evaluation researchPositive2:29:18

    Low-compute researchers can maximize impact by pursuing narrow evaluations of frontier-model weaknesses.

    “if you want to scope the maximum possible impact with minimum compute, it's something like that which is just get very narrow”

    Listen at 2:29:18

  43. Nathan Lamberton OpenAI employee compensationPositive2:30:10

    OpenAI’s average annual employee compensation exceeds one million dollars in stock.

    “The amount of OpenAI's average compensation is over a million dollars in stock a year per employee.”

    Listen at 2:30:10

  44. Sebastian Raschkaon Text diffusion modelsMixed2:47:25

    Text diffusion models will complement rather than replace autoregressive LLMs.

    “I don't think the text diffusion model is going to replace auto regressive LLMs but it will be something maybe for quick, cheap at scale tasks.”

    Listen at 2:47:25

  45. Sebastian Raschkaon Text diffusion modelsNegative2:47:25

    Text diffusion models will not replace autoregressive LLMs.

    “the text diffusion model is going to replace auto regressive LLMs”

    Listen at 2:47:25

  46. Sebastian Raschkaon Text diffusion modelsPositive2:47:29

    Text diffusion models will serve fast, inexpensive, high-volume tasks.

    “it will be something maybe for quick, cheap at scale tasks.”

    Listen at 2:47:29

  47. Sebastian Raschkaon LLM tool usePositive2:49:35

    Tool use reduces, but does not eliminate, LLM hallucinations.

    “Not solve it, but reduce it.”

    Listen at 2:49:35

  48. Nathan Lamberton AI automation of remote workPositive2:53:52

    A key AI milestone is replacing remote workers performing digital information tasks.

    “a key milestone among the AI community is essentially when AI could replace any remote worker taking in information and solving digital tasks and doing them.”

    Listen at 2:53:52

  49. Nathan Lamberton AI context windowsPositive2:59:33

    AI context windows will reach two to five million tokens in 2026.

    “I would expect it to keep increasing and get to 2 million or 5 million this year.”

    Listen at 2:59:33

  50. Nathan Lamberton RoboticsMixed3:13:05

    Home learning robots are unattractive, while self-driving and industrial automation are promising.

    “I'm so bearish on in home learned robots for consumer purchase. I'm very bullish on self driving cars and I'm very bullish for robotic automation.”

    Listen at 3:13:05

  51. Nathan Lamberton Self-driving cars and robotic automationPositive3:13:11

    Self-driving cars and industrial robotic automation have strong prospects.

    “I'm very bullish on self driving cars and I'm very bullish for robotic automation.”

    Listen at 3:13:11

  52. Nathan Lamberton AI research automationNegative3:19:43

    AI will probably not automate AI research within the discussed timeframe.

    “I would say probably not, at least in this timeframe.”

    Listen at 3:19:43

  53. Nathan Lamberton Software development automationPositive3:20:19

    Software automation will increase dramatically by the end of 2026.

    “By the end of this year, the amount of software that'll be automated will be so high.”

    Listen at 3:20:19

  54. Nathan Lamberton Software engineering workNeutral3:21:12

    AI will shift software engineering toward system design and outcome specification.

    “software engineering will be driven more to system design and goals of outcomes”

    Listen at 3:21:12

  55. Nathan Lamberton Software development workNeutral3:23:51

    Software workers will soon shift toward design and product direction.

    “within like on the order of low years, a lot of people are going to be pushed to be more of like a designer and product manager”

    Listen at 3:23:51

  56. Nathan Lamberton AI computer useNegative3:32:44

    AI computer-use systems have made limited progress despite major lab attention.

    “Computer use is a good example of what labs care about and we haven't seen a lot of progress on.”

    Listen at 3:32:44

  57. Nathan Lamberton Computer-use AI systemsNegative3:32:49

    Current computer-use systems from Claude and OpenAI perform poorly.

    “We saw multiple demos in 2025 of like Claude can use your computer or OpenAI had CUA and they all suck.”

    Listen at 3:32:49

  58. Nathan Lamberton AI computer useMixed3:36:41

    Software automation may advance substantially while computer use requires additional innovation.

    “We might get this software solution, but it might stop at software and not do computer use without more innovation.”

    Listen at 3:36:41

  59. Nathan Lamberton Deep learning scalingPositive3:37:10

    If scaling laws remain fundamental, compute scaling will continue to dominate progress.

    “If scaling laws are fundamental and deep learning, I think the bitter lesson will always apply”

    Listen at 3:37:10

  60. Sebastian Raschkaon AI capability progressPositive3:41:21

    AI capability improvements will continue through amplification rather than paradigm change.

    “I can see that continuing for a long time.”

    Listen at 3:41:21

  61. Nathan Lamberton AI assistantsNeutral3:47:05

    AI assistants will eventually be monetized through advertising.

    “they're going to be paid for by ads. It's coming.”

    Listen at 3:47:05

  62. Nathan Lamberton Google AI advertisingPositive3:49:19

    Google will likely develop effective AI advertising sooner than competitors.

    “Google is probably going to be better at figuring out how to do this”

    Listen at 3:49:19

  63. Nathan Lamberton AI industryNeutral3:52:10

    Consolidation in the AI industry will continue.

    “this trend of consolidation will continue.”

    Listen at 3:52:10

  64. Nathan Lamberton AI API marketNegative3:57:28

    Unprofitable AI APIs may force providers into products and hardware.

    “the API market is so unprofitable that they go up and down the stack to products and hardware.”

    Listen at 3:57:28

  65. Sebastian Raschkaon LlamaPositive3:59:48

    Llama’s proper goal is usable, trustworthy, modifiable models including smaller versions.

    “the goal was to have a model that people can use, trust, modify, understand that includes having smaller models.”

    Listen at 3:59:48

  66. The United States should invest in building leading open AI models.

    “the US should be building the best models so that the best research happens in the US”

    Listen at 4:05:12

  67. Nathan Lamberton US open-model restrictionsNegative4:10:44

    Banning globally trained open models would require an impractical US internet firewall.

    “I think effectively that's impossible without making the US have its own great firewall”

    Listen at 4:10:44

  68. Nathan Lamberton Open AI modelsPositive4:12:34

    Open models will eventually dominate if progress saturates within several years.

    “open models will be so optimized and so much cheaper to run that they will win out.”

    Listen at 4:12:34

  69. Sebastian Raschkaon Advanced AI knowledgeNegative4:13:57

    Governments cannot effectively contain advanced AI knowledge.

    “I don't think there is a way you can contain something like that, like knowledge like that.”

    Listen at 4:13:57

  70. Sebastian Raschkaon Nvidia CUDA ecosystemPositive4:15:13

    Nvidia’s durable advantage is primarily its CUDA ecosystem, not chips alone.

    “the mode of Nvidia is, is probably not just the gpu, it's more like the Cuda ecosystem”

    Listen at 4:15:13

  71. Nvidia’s Jensen-centered operating culture supports continued competitive progress.

    “So long as that is how it operates. I'm pretty optimistic for their situation”

    Listen at 4:17:57

  72. Nvidia is likely to maintain strong prospects under Jensen Huang’s leadership.

    “I'm pretty optimistic for their situation”

    Listen at 4:17:59

  73. Nathan Lamberton ChatGPT-like AI developmentMixed4:19:14

    ChatGPT-like AI progress would have occurred without Jensen, but more slowly.

    “eventually something like ChatGPT would have happened and a build out like this would have happened but it probably would not have been as fast”

    Listen at 4:19:14

  74. Sebastian Raschkaon Specialized robotsPositive4:27:19

    Specialized robots will become widespread for particular tasks.

    “I do think specialized robots for sure, for certain tasks.”

    Listen at 4:27:19

  75. Nathan Lamberton PCNeutral4:28:38

    People will still carry personal computing devices in the future.

    “I still expect to have people carry things around.”

    Listen at 4:28:38

  76. Universal basic income will not by itself solve human agency.

    “I think that UBI does not Solve agency.”

    Listen at 4:30:03

  77. AI-generated content will increase the premium on in-person human experiences.

    “my hope for all the AI slop we're seeing is that there will be a greater and greater premium for the fundamentals aspects of the human experience that are in person.”

    Listen at 4:31:37

  78. Nathan Lamberton Physical goods and eventsPositive4:31:55

    Physical goods and events will become more valuable as AI slop increases.

    “The next few years are definitely going to be an increased value on physical goods and events and even more pressure on slop.”

    Listen at 4:31:55

  79. Sebastian Raschkaon ArtPositive4:32:35

    Art will persist despite increasing automation and synthetic media.

    “I don't think art will go away.”

    Listen at 4:32:35

  80. AI-generated content will eventually fool people, requiring trust-building platforms.

    “I think eventually they'll fool you and it'll be on platforms that give ways of verifying or building trust.”

    Listen at 4:33:41

  81. Sebastian Raschkaon AI-generated content authenticationPositive4:34:01

    Distinguishing AI from human content is solvable through trusted outlets.

    “fundamentally, I think it's a solvable problem by having, you know, trust in certain outlets that they won't do it.”

    Listen at 4:34:01

  82. Nathan Lamberton homo sapiensPositive4:35:39

    Humans tend to find solutions to major problems.

    “humans do tend to find a way.”

    Listen at 4:35:39

  83. Nathan Lamberton AI and related technologiesPositive4:35:51

    AI and related technologies offer substantial opportunities.

    “the AI opportunity in related technologies is really big.”

    Listen at 4:35:51

  84. Nathan Lamberton Lasting AI benefitsMixed4:36:23

    Achieving lasting AI benefits is probably feasible but will take time.

    “It is something that is probably doable, just will take longer than people want.”

    Listen at 4:36:23

  85. Sebastian Raschkaon Current AI systemsNegative4:37:20

    Current AI cannot independently determine changes and requires human instructions.

    “AI in its current implementation can't see it changing. You have to tell it what to do.”

    Listen at 4:37:20

  86. Sebastian Raschkaon Current AI systemsNeutral4:37:32

    Current AI functions as a tool directed by humans.

    “it becomes a tool. You can think of it as a tool.”

    Listen at 4:37:32

  87. Sebastian Raschkaon Current AI systemsPositive4:37:46

    Humans remain in control of current AI systems.

    “the AI is not in charge. You're in charge.”

    Listen at 4:37:46

  88. Sebastian Raschkaon AI systemsNegative4:38:08

    AI could cause harm if explicitly programmed to do harmful things.

    “The only thing I can see going wrong is of course, if things are explicitly programmed to do the thing that is harmful, basically.”

    Listen at 4:38:08

  89. Lex Fridmanon Humans versus machinesPositive4:38:18

    Humans will win a Terminator-like war against machines.

    “I think humans win.”

    Listen at 4:38:18

  90. Lex Fridmanon Local and open-source LLMsPositive4:38:30

    Humans will likely use local and open-source LLMs against machines.

    “we'll probably be using local LLMs, open source LLMs to help fight the machines.”

    Listen at 4:38:30

Statements are attributed to the speaker as said on the episode and reflect their view at the time, not PodLume's. They are not advice.

Weekly email

State of the Pod, every Monday

The week's most-discussed names and stories from podcasts, with the moments behind them. Free, five minutes, unsubscribe in one click.

Listen to the full episode and explore every guest, topic, and moment on PodLume.