Nov 8, 2026 · 59 min
Frontier AI turns product judgment into a technical discipline
What it actually takes to build frontier models | Tulsee Doshi (Head of Product for Gemini)
As models evolve unpredictably, product teams must connect research, evaluations, safety, and user experience without betting everything on capabilities that may change.
- 1Frontier model progress emerges from research, training, evaluations, product design, and safety rather than engineering alone.
- 2AI-native PMs need model intuition, rigorous evaluations, technical scrappiness, and judgment about what deserves to be built.
- 3Flexible architectures and a 60-20-20 roadmap help products deliver value today while preparing for rapidly improving models.
Don't miss
Doshi explains why replacing the Fitzpatrick scale with the Monk Skin Tone Scale exposed evaluation as a product and representation problem.
The brief
Tulsee Doshi describes frontier model development as an unpredictable loop among research, training, evaluations, safety, and product design—not a straightforward engineering pipeline.
Google’s full-stack strategy spans hardware, models, products, and billions of users, while offering systems tuned for different costs, capabilities, and use cases.
The model itself has a UX: its transparency, communication style, personality, and surprising capabilities shape whether a product feels useful or confusing.
Doshi’s advice for PMs combines hands-on model intuition with structured evals, starting from a small set of meaningful prompts and a clear definition of good.
Her 60-20-20 framework keeps products grounded: make most of the experience work now, reserve some for near-term gains, and leave room for ambitious capabilities.
The conversation’s sharpest tension is between AI’s ability to accelerate output and the human judgment needed to prevent low-value products and AI slop.
What was said on this episode
14 statements · 9 positive · 2 negative · 3 neutral
- Tulsee Doshion Google AI product UXPositive15:30
Google should experiment with new paradigms and UX approaches for model interaction.
“we do need to be playing with new paradigms and new UX approaches”
Listen at 15:30
- Tulsee Doshion AI product UXNeutral16:52
AI product UX should vary according to the target user.
“depending on who your user is, you have a certain level of flexibility with how you think about the UX”
Listen at 16:52
- Tulsee Doshion AI model UXNeutral17:06
AI models themselves have a user experience shaped by transparency and communication.
“the model as having a UX also”
Listen at 17:06
Gemini behavior development should incorporate customer and user feedback.
“we've been thinking about together is A, looking at customer feedback and user feedback”
Listen at 18:00
Gemini behavior specifications can be translated into evaluation rubrics.
“translate that to rubrics, right, that we can then evaluate the model on”
Listen at 18:15
Agent systems can replicate flash-team principles from crowdsourced human teams.
“now we're kind of able to replicate that world with agents”
Listen at 32:00
- Tulsee Doshion Sub-agent systemsPositive32:09
Flash-team principles can guide the design of effective sub-agent systems.
“how do we apply some of the same principles, if you will, but to like how you actually set up an effective system of sub-agents”
Listen at 32:09
Flash teams can mobilize individuals into highly effective temporary teams.
“the whole notion of a flash team and this idea that you can actually like mobilize individuals into something that can be highly effective”
Listen at 32:39
- Tulsee Doshion Workplace burnoutNeutral45:35
Burnout is not caused solely by working long hours.
“burnout comes not just from working a lot”
Listen at 45:35
- Tulsee Doshion Workplace burnoutNegative45:42
Workplace churn and chaos contribute to burnout.
“it comes, one, from when you feel like there's churn and chaos”
Listen at 45:42
- Tulsee Doshion Workplace burnoutNegative45:50
Unfulfilling work contributes to burnout.
“work that you don't enjoy”
Listen at 45:50
- Tulsee Doshion AI productivity toolsPositive46:20
AI tools may reduce burnout by removing grunt work and frustrating tasks.
“these tools actually help with burnout because they actually remove some of the like. grunt work or the like frustrating parts of your job”
Listen at 46:20
- Tulsee Doshion Product-management workPositive47:14
Removing friction from PM work may reduce burnout risk.
“how do we pull out the friction parts of their work? that are likely to induce burnout”
Listen at 47:14
- Tulsee Doshion Employee motivationPositive47:35
Organizational leaders should account for each employee’s individual motivations.
“part of it for me is like an org leader also is to think about what motivates the individual”
Listen at 47:35
Statements are attributed to the speaker as said on the episode and reflect their view at the time, not PodLume's. They are not advice.
Books & mentions
Some links are affiliate links — PodLume may earn a commission if you buy.
Weekly email
State of the Pod, every Monday
The week's most-discussed names and stories from podcasts, with the moments behind them. Free, five minutes, unsubscribe in one click.