← Moonshot AI K3 attention mechanism
What podcasts say about Moonshot AI K3 attention mechanism
Every statement, with the speaker, the exact quote and the moment it was said.
What experts have said about Moonshot AI K3 attention mechanism
2 statements · 2 positive
Kimi K3’s NOPE mechanism gives more weight to recent context than distant context.
“But nope. Actually has this fading memory now. So something I said a long time ago gets less weight than something that's more recent.”
Open the episode · Dario vs Jensen on Open Weights, OpenAI & Anthropic in DC, Xi Exports AI to Global South | EP #275Listen at 39:49
K3’s attention changes reduce memory usage by 75%.
“Those attention mechanism changes cut the memory use by 75%”
Open the episode · The Hugging Face Breach, Moonshot AI Valued at $20B, and Living to 1,759 Years Old | EP #273Listen at 18:40
Statements are attributed to the speaker as said on the episode and reflect their view at the time, not PodLume's. They are not advice.