Why Expert Predictions So Often Fail
And why that tells us less about expertise than we think
A much shorter, and less refined version of this post was published on Psychology Today.
You can also hear AI Matt’s summary of the piece below.
A while back, I came across an article titled “Trust the Experts? It’s a Bad Bet.” At first glance it sounded like it was going to echo an argument I myself made concerning the so-called “War on Expertise.” But reading past the headline revealed a much narrower—and very different—argument. The real target wasn’t expertise in general; it was expert prediction. The claim, in essence, was that experts are often poor forecasters of the future, and trusting their predictions is usually a losing proposition.
On that point, the author is largely right. Trusting expert predictions often is a bad idea, as Grossman et al. (2023)1 recently showed. But it’s not because expertise itself is overrated—at least not real expertise. It’s because what makes someone a true expert is rarely their ability to function as a prophet.
Judging expertise by its ability to predict uncertain futures almost guarantees disappointment—and it sets up a false dichotomy in the process: either we defer to expert authority, or we conclude that expertise itself is overrated. Neither position gets us very far.
The more interesting question isn’t whether we should “trust experts” in some broad, abstract sense. It’s what we should expect expertise to actually deliver—and where its limits lie. True expertise helps us make sense of the present: to diagnose problems we already face, interpret evidence we already have, and weigh tradeoffs under real constraints. When we confound that kind of judgment with prophecy, we misunderstand what expertise is for in the first place.
What Expertise Actually Is
Expertise isn’t primarily about long-horizon prediction. It’s about judgment—specifically recognizing what kind of situation you’re in and responding in ways that have worked before under similar conditions (aka recognition-primed decision making). Across domains, what distinguishes experts from novices isn’t superior logic or broader abstract knowledge—it’s their experience with recurring patterns. Over time, experts learn which cues matter, which ones can be ignored, and which tradeoffs are likely to follow from different courses of action—ultimately giving rise to what Kahneman & Klein (2009) described as skilled intuition.
When faced with a problem, experts don’t start from scratch by weighing every possible option. Instead, they recognize familiar elements, generate a small set of plausible responses, and then evaluate whether those responses fit the specifics of the situation at hand. When the context aligns with their prior experience, this process can be remarkably fast and effective. It’s also why expert intuition can look effortless from the outside: much of the work has already been done through repeated exposure to similar problems (Prietula & Simon, 1989).
That strength, however, comes with a clear limitation. Expert judgment is tightly bound to context. It works well within what I’ve referred to elsewhere as an expertise “bubble”—the range of situations in which expert judgment provides reliable guidance.
That doesn't mean expertise is blind to the near future. A veteran firefighter can anticipate how a blaze will likely move. An experienced physician can foresee likely complications. But those judgments unfold in environments where patterns recur and feedback arrives quickly. As problems drift farther beyond that bubble—toward novel conditions, unfamiliar environments, or extended time horizons—the cues that once anchored recognition begin to disappear. At that point, judgment shifts from recognition to extrapolation (a pattern echoed in Grossman et al., 2023). And extrapolation is a far shakier endeavor.
Why Expert Predictions Often Look Like a Coin Toss
Imagine an expert is brought into an organization to help solve a problem. They’re given a general description of what’s going wrong and then asked, on the spot, to say what the organization should do. They’re given no opportunity to dig deeper and no chance to gather additional information. They’re simply asked to make a judgment call.
In that situation, even a genuine expert has little choice but to guess. Some of the assumptions they make will be reasonable. Others won’t. And unless the problem is unusually simple or well-structured, there’s no reason to expect the final judgment to be much better than chance. That isn’t a knock on expertise; it’s what happens when judgment is forced to operate without the evidence it needs to do its job.
What differentiates experts in real settings isn’t their ability to issue answers on demand. It’s their ability to shape the judgment process based on their skilled intuition—to guide the collection of evidence in ways that make judgment possible. They know which questions to ask next, which information is worth gathering, and when additional data will no longer meaningfully improve their decision. That process refines and calibrates the pattern recognition that underlies expert judgment.
Stretch that judgment into longer-term forecasting, and the process begins to short-circuit. By definition, the problem is thin on direct evidence and saturated with uncertainty. There’s little opportunity for guided information gathering, iteration, or meaningful feedback. The expert is forced to fill in the gaps with assumptions about how people, organizations, or systems will behave. Some of those assumptions will be right, and others won’t. By the time events unfold, the original prediction is rarely revisited in detail, the assumptions that supported it are almost never examined, and apparent successes may reflect luck rather than sound judgment. Averaged out, accuracy drifts toward a coin toss.
This is why expert predictions so often disappoint. It’s not that experts suddenly lose their competence when asked about the future. It’s that prediction removes the very mechanism that makes expertise useful in the first place: context-sensitive judgment refined through selective evidence and feedback over time. When we strip that away, what remains isn’t expert insight so much as guesswork—and the results look exactly like you’d expect.
Why We Keep Asking Experts to Predict Anyway
If expertise doesn’t translate well into making oracle-like predictions, it begs the question of why we keep asking experts to do it in the first place. There are several reasons this practice persists; here I want to focus on three that are closely related2.
The first is a basic but persistent confusion about what expertise actually entails. Expertise involves informed judgment—recognizing relevant cues, interpreting evidence, and weighing tradeoffs in context. Because experts are good at making sense of complex situations, it’s easy to assume that skill should extend naturally to saying what will happen next. In practice, those are different tasks operating under very different conditions (Önkal et al., 2003), but the distinction is rarely made explicit.
The second is what might be called prediction theater. The future is, by definition, uncertain, but that uncertainty will eventually resolve into events that affect people’s lives—creating opportunities, producing risks, and forcing choices with real consequences. Asking experts to forecast those events offers a way to make an open-ended future feel more manageable. A forecast compresses uncertainty into a concrete claim about what’s likely to happen, even when the underlying uncertainty remains unchanged.
The third is that predictions are unusually difficult to evaluate. Many forecasts won't work out, but prediction accuracy often won't be clear until much later. By then, few people remember the original forecast in enough detail to evaluate its accuracy, and fewer still revisit the assumptions that supported it. Even apparent successes can be misleading, since it's often difficult to determine whether a prediction was correct because of sound judgment or simple luck.
This makes forecasting unusually difficult to evaluate and creates a system that selectively amplifies confident predictions while rarely holding them accountable. Experts who speak decisively are rewarded with attention and credibility, while those who hedge, qualify, or emphasize uncertainty are often sidelined (Lawrence et al., 2006). The result is a forecasting environment that rarely learns from either success or failure.
What Expertise Is (and Isn’t) Good For
None of this means expertise is useless, or that skepticism toward experts is misplaced. It means we often ask experts to do the wrong job. Expertise isn’t a crystal ball. It’s a way of making sense of messy situations in the present—of diagnosing problems, interpreting evidence, and helping decision makers navigate tradeoffs under real constraints.
When used that way, real expertise still matters a great deal. Experts are most valuable when they’re allowed to interact with a problem: to ask questions, gather targeted evidence, test assumptions, and revise their judgments as new information comes in. That’s where recognition-primed judgment shines—and where expert insight genuinely improves decisions.
Trouble starts when we collapse that judgment process into a demand for prediction. When we treat experts as forecasters of uncertain futures, we strip away the context and evidence their judgment depends on, then fault them when accuracy collapses. The result is predictable disappointment, followed by an unhelpful backlash against expertise itself.
That doesn’t mean some people don’t outperform others at prediction, but those differences reflect strategies for managing uncertainty, not a deeper form of expertise or foresight. And even then, reliability tends to decay as the time horizon expands3.
If we want better decisions, the solution isn’t to stop listening to experts—or to trust them blindly. It’s to stop confusing judgment with prophecy. We should expect expertise to deliver what it actually can: sound judgment about the situation in front of us, not reliable predictions once we move beyond it.
I thank Lee Jussim for linking to this article in one of his X posts.
Mental Garden (2025) offers a similar argument as I do here, and it’s worth checking out.
While structured forecasting disciplines can improve calibration under narrow, short-horizon conditions, those gains are often modest and sensitive to question design and scoring methods. I discuss these issues in more detail in The Limits of Rationalism.
The part I would push past where you leave it is the third reason, why we keep asking even after we know it fails. You frame it as confusion plus theater plus poor evaluability. I think there is a fourth sitting underneath those, and it is not epistemic at all.
A prediction moves responsibility. A judgment about present conditions keeps the decision in the room, shared by everyone in it. A forecast lets the uncertainty be placed in one person's mouth, and once it is there, the rest of the room is lighter. That may be the real reason the decisive forecaster is rewarded and the one who hedges gets sidelined. It is not only that confidence reads as competence. It is that the confident forecaster is accepting a transfer the room is trying to make, and the hedger is refusing to carry it.
Which reframes your accountability problem. The forecasting environment does not just fail to learn because predictions are hard to score. It resists learning because the social function was never accuracy. It was relief. A system built to move a burden has little reason to check later whether the burden was placed correctly, and every reason to keep rewarding whoever will take it on demand.