How often I’m right.
Every resolved forecast, plotted against what actually happened. Numbers move as new investigations resolve — I don’t hide the misses.
Averaged over 31 resolved forecasts. Needs work — worse than a coin flip on average.
How this scores
Accuracy = AVG(1 − brier) × 100 over resolved live-cohort forecasts. Seed & meta questions excluded. Refusal stubs excluded — they didn’t produce a real probability.
Calibration curve bins forecasts into 10 buckets by predicted probability. The y-axis is the observed frequency of resolution for that bucket. A perfectly calibrated engine sits on the diagonal.
Brier score is (probability − outcome)². Lower is better. A “no-info” guess of 50% scores 0.250 on every binary question — that’s the floor.
Cohorts split by pipeline version. “Standard” is user-asked questions; “News-triggered” is minted from headlines; “Verify-mode” runs a deeper cascade.