
Sep 16, 2026 · 25 min
Judgment models push AI beyond text generation
Why a New Class of AI “Judgment Models” Could Have Big Business Implications
Fast probabilistic assessments could give AI agents a cheaper way to verify work and coordinate consequential business decisions.
- 1Jev represents an AI model class that answers targeted questions with probabilities rather than generating long-form text.
- 2Judgment models could help agents check their own work and support coordination across business processes.
- 3The episode places Jev alongside debates over AI regulation, Zuckerberg’s opposition to a collective slowdown, and Salesforce’s expanding agent ecosystem.
Don't miss
Whittemore’s discussion of Jev frames probabilistic judgment—not text generation—as a possible foundation for agents that verify their own work.
The brief
Nathaniel Whittemore introduces judgment models, a category aimed at answering specific questions with probabilities instead of producing long-form prose.
The episode focuses on Jev, a model from Typesafe designed to deliver fast, inexpensive assessments that could make AI systems more useful in practical settings.
The central business question is whether agents can use judgment models to verify their work, assess uncertainty, and coordinate decisions without relying on another lengthy generation.
Jev’s implications sit within a broader AI landscape shaped by regulation debates, Zuckerberg’s opposition to a collective slowdown, and Salesforce’s expanding agent ecosystem.
The standout idea is that AI progress may increasingly depend on compact systems that evaluate options and confidence, not just models that produce more text.
What was said on this episode
20 statements · 13 positive · 5 negative · 2 mixed
Typesafe’s Jev produces probabilities for specific questions rather than long-form text.
“these judgment models like the one we're discussing today, Jev from Typesafe, produce probabilities around specific questions”
Listen at 0:18
Judgment models can produce judgments faster and more cheaply than conventional approaches.
“how they can produce these judgments much more quickly and much less expensively”
Listen at 0:35
Nathaniel predicts AI policy will become more specific and nuanced.
“the next phase is likely to include a lot more specificity, and dare I say nuance”
Listen at 7:04
Confusing AI political discourse may indicate policy progress
“I think that the more that the political discourse kind of frags your brain for how confusing and all over the place it is, that might just be a sign that we're actually making progress.”
Listen at 7:20
Increasingly confusing AI political discourse may indicate progress.
“the more that the political discourse kind of frags your brain for how confusing and all over the place it is, that might just be a sign that we're actually making progress”
Listen at 7:21
Salesforce AI Force enables third-party agents to serve as software interfaces.
“allowing any agent to become the interface”
Listen at 8:44
Jev cannot generate text or replace general-purpose LLMs
“this model does not generate text. It is not, in other words, a replacement for LLMs in general.”
Listen at 15:31
Jev does not generate text.
“this model does not generate text”
Listen at 15:31
Jev answers narrowly defined questions using probabilities, categories, or scores.
“Jev then is designed to take the relevant information and answer narrowly defined questions with probabilities, categories, or scores”
Listen at 16:17
Cheap judgment models could check every request and draft before consequential agent actions.
“it could run on every incoming request after each draft revision across many candidate documents before an agent takes a consequential step”
Listen at 18:42
Jev can apply conditional workflow logic to messy human context.
“Jev is basically asking, what if those if statements could understand messy human context?”
Listen at 20:16
Nathaniel expects judgment models to be useful across fraud, support, moderation, compliance, and routing.
“I can see this being very useful for fraud and risk, support routing, moderation, PR and QA automation, lead scoring, compliance, workflow orchestration, and agent routing”
Listen at 20:24
LLMs are often unsuitable and potentially inferior to classic machine-learning methods for classification tasks.
“it's often the wrong tool for the job, and possibly not as good as classic ML methods for what they are trying to do”
Listen at 21:17
Nathaniel predicts judgment models will excel in teamwork-oriented workflows.
“where this sort of technique is going to really shine is in the domain of teamwork”
Listen at 22:14
Judgment models cannot perform all tasks currently handled by generative AI.
“The big cost of JEV and this type of judgment model in general, to the extent that this becomes a category, is that it is an incomplete category by definition.”
Listen at 23:48
Judgment models cannot perform all tasks currently handled by generative AI.
“it cannot do all the work that we currently have generative AIs do”
Listen at 23:57
Judgment models can perform their specialized tasks at very low cost.
“The benefit, of course, though, is that it can do this extraordinarily inexpensively.”
Listen at 24:10
Low-cost judgment intelligence can be deeply integrated into automated systems.
“because this sort of judgment intelligence can be applied so cheaply, it means that done well, it can be integrated incredibly deeply into the automated systems we're all building”
Listen at 24:14
Low-cost judgment intelligence can be deeply integrated into automated systems.
“it means that done well, it can be integrated incredibly deeply into the automated systems we're all building”
Listen at 24:18
Nathaniel assesses judgment models as important and conceptually obvious.
“it does have the feel when you dig in of something both important and obvious”
Listen at 24:34
Statements are attributed to the speaker as said on the episode and reflect their view at the time, not PodLume's. They are not advice.
Weekly email
State of the Pod, every Monday
The week's most-discussed names and stories from podcasts, with the moments behind them. Free, five minutes, unsubscribe in one click.