← Long-context language models
What podcasts say about Long-context language models
Every statement, with the speaker, the exact quote and the moment it was said.
What experts have said about Long-context language models
2 statements · 2 negative
Memory bandwidth and capacity primarily limit very large context windows.
“the primary thing that limits you to have really large contexts are memory bandwidth, memory capacity”
Open the episode · Reiner Pope – The math behind how LLMs are trained and servedListen at 1:53:02
Expanding context windows far beyond roughly 100,000 tokens would be prohibitively expensive.
“going massively beyond that would be cost prohibitive”
Open the episode · Reiner Pope – The math behind how LLMs are trained and servedListen at 1:54:23
Statements are attributed to the speaker as said on the episode and reflect their view at the time, not PodLume's. They are not advice.