Everyone gets things wrong sometimes. But when people with prestigious titles, decades of experience, and access to classified information all consistently overestimate their own abilities? That's when things get interesting. A massive study tracked nearly 300 experts making over 82,000 forecasts, and the results should make you feel slightly better about your own bad takes.
The research found that these professionals were systematically overconfident. Events they called "sure things" failed to materialize about 25% of the time, while events they dismissed as "impossible" happened roughly 15% of the time. Being an expert, it turns out, does not make you psychic.
The Numbers Don't Lie And Neither Does The Humiliation
Philip Tetlock led the 20-year study, which became one of the most cited pieces of research on human judgment. He asked experts across multiple fields to make forecasts about political, economic, and geopolitical events and then tracked what actually happened.
The results were not kind. The experts performed barely better than if they'd flipped a coin on long-term predictions. Having advanced degrees, specialized training, or connections in high places did not meaningfully improve accuracy. The overconfidence bias — that mental glitch that makes us certain right before we get proven spectacularly wrong — was alive and well across every discipline studied.
But the story doesn't end there.
Meet The Superforecasters
Tetlock didn't just document failure. He also found a small group of regular people who consistently outperformed the professionals. Teachers, retired plumbers, accountants — people without special credentials or secret information. He called them "superforecasters."
These individuals shared certain habits. They gathered information from multiple opposing sources rather than seeking confirmation of what they already believed. They updated their views frequently as new data came in. They avoided absolute language and thought in probabilities rather than certainties.
Research on the Dunning-Kruger effect later added context: people with limited knowledge in a domain often have the highest confidence about it. The less someone understands a complex situation, the simpler the future looks to them.
Why It Matters
Here's the uncomfortable truth: we are all walking around assuming we're sharper than average. Studies consistently show that most people rate themselves as better-than-average drivers, better-than-average managers, and significantly better than "others" at spotting misinformation. That inflated self-assessment is the same trap that tripped up the experts in Tetlock's study.
Research from the Behavioral Insights Team found that 59% of people felt confident in their ability to spot false information online — while only 36% believed others could do as good a job. Eva Kolker from BIT put it bluntly: overconfidence in spotting misinformation "might actually be making people more susceptible to it."
The superforecasters succeeded precisely because they fought against that instinct. They stayed humble. They treated their predictions as hypotheses to be tested rather than truths to be defended. It's a useful reminder that good judgment isn't a personality trait — it's a practice.
A Worked Example
Say you see someone on social media making a definitive prediction about next quarter's economic performance. They sound certain. They have charts. They have followers. The superforecaster approach would be to ask: what sources are they using? Are those sources cherry-picked? What would have to be true for them to be wrong? Probability, not certainty.
What To Watch Next
Tetlock's findings have spawned practical applications. The US intelligence community now runs competitions to improve forecasting methods, and organizations train people in probabilistic thinking. Whether that trickles down to everyday confidence levels is another question. For now, the lesson stands: next time someone tells you something is guaranteed to happen, maybe hold your applause.
Who Should Care
If you're a decision-maker:** Overconfidence has real costs. The superforecaster habits — diverse sourcing, probabilistic thinking, frequent updating — can be systematically taught and improve outcomes.
If you're a consumer of news:** That voice screaming about "guaranteed" outcomes? The data says they're wrong roughly a quarter of the time. Treat certainty as a warning sign, not a signal of competence.
If you're just trying to not look foolish on the internet:** The same bias that fooled credentialed experts is inside you. Humility isn't weakness; it's accuracy calibration.
Key takeaways
- A 20-year study found that expert predictions were barely better than random chance over the long term.
- Events experts called "sure things" failed 25% of the time; "impossible" events happened 15% of the time.
- A small group of everyday people, called "superforecasters," consistently beat professionals by using specific habits.
- Superforecasters gather diverse sources, update views often, and avoid absolute language.
- Research shows overconfidence in spotting misinformation may actually increase susceptibility to being fooled.
Frequently asked questions
What was the name of the study that tracked expert predictions?
It was a 20-year study led by Philip Tetlock, which tracked over 82,000 forecasts from nearly 300 experts across political, economic, and geopolitical topics.
How did experts perform compared to random chance?
Their long-term predictions were barely better than a coin flip, and they were systematically overconfident about their own accuracy. Events labeled "sure things" failed 25% of the time, while "impossible" events occurred 15% of the time.
What are superforecasters?
Superforecasters are everyday people — like teachers, accountants, and retired tradespeople — who consistently outperformed credentialed experts by using specific thinking habits: gathering diverse opposing sources, updating views as new data arrives, and avoiding absolute language in favor of probabilities.
What is the Dunning-Kruger effect?
It's a cognitive bias where people with limited knowledge in a subject often hold the highest confidence levels, because they don't yet understand how complex the situation actually is. The less we know, the simpler the future looks.
Why do people enjoy stories about experts being wrong?
It validates a universal experience: everyone has been overconfident at some point. Schadenfreude about expert failures is really just relief that we're not alone in being wrong — and perhaps a reminder to stay humble about our own certainties.