Measure the cognitive skills that actually predict forecasting ability
Intelligence matters for many life outcomes, but surprisingly, IQ only weakly predicts forecasting ability (r ≈ .2). Even more counterintuitively, Tetlock's 20-year study of 82,361 predictions found that specialists performed no better in their own field than outside it—and "hedgehog" experts who relied heavily on domain expertise actually did worse when predicting within their specialty. Smart experts get caught off guard because expertise breeds overconfidence, not accuracy.
What matters more is what Mellers et al. called "good judgment"—a combination of probabilistic thinking, calibration, and willingness to update beliefs. This isn't an IQ test. It measures the cognitive skills that actually predict whether you'll be good at anticipating the future.
1. Take the Assessment — Measure your baseline across Bayesian reasoning, diagnostic thinking, cognitive reflection, and open-minded thinking.
2. Make Predictions — Forecast real-world events drawn from prediction markets. Your predictions are stored and scored when events resolve.
3. Track Your Accuracy — See how your judgment score correlates with actual forecasting performance over time.
Based on research from Tetlock, Mellers, and Baron
This measures five dimensions of judgment quality that predict forecasting accuracy: Bayesian updating, regression awareness, calibration, cognitive reflection, and open-minded thinking.
Time: ~8-10 minutes
Choose a display name for the leaderboard. This is optional—you can stay anonymous if you prefer.
Retake the assessment with a fresh start
Forecast real events. Your accuracy will be tracked and compared to your judgment score.
Instructions: For each question, drag the slider to your probability estimate. The market price is shown for reference—you're welcome to agree or disagree with it.
Predictions are scored using the Brier score when events resolve. Lower is better.
No predictions yet.
Tracking the correlation between judgment scores and forecasting accuracy
Mellers et al. (2017) found that superforecasters' judgment scores (a composite of Bayesian reasoning, diagnostic thinking, and other measures) correlated r ≈ .46-.60 with their forecasting accuracy.
We're testing whether this holds in the wild. As predictions resolve, we'll report the correlation between assessment scores and Brier scores.
Lower Brier scores indicate better prediction accuracy. Scores range from 0 (perfect) to 1 (worst).
| Rank | User | Brier Score | Judgment Score | Predictions |
|---|---|---|---|---|
| Waiting for predictions to resolve... | ||||
The science behind the assessment
This project tests whether laboratory measures of judgment quality predict real-world forecasting accuracy. The assessment is based on two major research programs.
From 2011-2015, Philip Tetlock and Barbara Mellers ran an IARPA-sponsored forecasting tournament with 5,000+ participants. "Superforecasters"—the top 2%—outperformed professional intelligence analysts by roughly 30%, even though the analysts had access to classified information.
A follow-up study asked whether superforecasters' skills generalized to other judgment tasks. They outperformed on Bayesian reasoning (40-78% vs 5-28% for undergraduates), diagnostic test selection (77% vs 54% on congruence bias), and showed better calibration.
We're testing whether that correlation replicates outside the lab. Participants take the assessment, make predictions on real events, and we track how judgment scores relate to forecasting accuracy as events resolve.
Yes. Unlike IQ, these skills appear trainable. GJP found that brief training improved accuracy by 10-15%, and superforecasters themselves improved over time. The key skills: calibration, base rate thinking, scope sensitivity, and systematic updating.
Your data is stored with an anonymous ID. We don't collect names or email addresses. You can bookmark your ID to return and track your predictions.