← Transformer Architecture
What podcasts say about Transformer Architecture
Every statement, with the speaker, the exact quote and the moment it was said.
What experts have said about Transformer Architecture
5 statements · 4 positive · 1 negative
The transformer architecture underlying ChatGPT came from Google Brain.
“The transformer, which sort of underpins everything, it's literally the T in ChatGPT. Yes, that came from Google Brain.”
Open the episode · Google X's Astro Teller: The $1B Bet No CEO Will Back, Moonshots 3x Cheaper in 16 Yrs, and Clean Water at 1¢/L| EP #300Listen at 8:20
Transformer attention technology will improve dramatically within the next year.
“we're going to see an explosion of that. This chart will be one of the first points that you see in that explosion of change that's going to come really in the next year.”
Open the episode · Should we slow down AI progress? | MOONSHOTS #288Listen at 1:33:14
Transformer attention mechanisms are inefficiently bloated.
“the whole attention mechanism is bloated”
Open the episode · Mira Murati's 975B Open Model, Ramin Hasani on Post-Transformer AI, and Demis' AI FINRA | EP #271Listen at 1:13:45
Transformers can process video, images, speech, and text in one architecture.
“you can feed it video or you can feed it images or speech or text, and it just gobbles it up”
Open the episode · #333 – Andrej Karpathy: Tesla AI, Self-Driving, Optimus, Aliens, and AGIListen at 39:03
The transformer is a powerful general-purpose differentiable computer.
“it's a general differentiable computer and it's extremely powerful”
Open the episode · #333 – Andrej Karpathy: Tesla AI, Self-Driving, Optimus, Aliens, and AGIListen at 45:43
Statements are attributed to the speaker as said on the episode and reflect their view at the time, not PodLume's. They are not advice.