Precogs.chat

Precogs Chat: predictive AI assistants, explained

A predictive assistant fits patterns in past data, extends them forward and attaches a range to its guess, so the only fair test is to score its forecasts against what happened. Public benchmarks show both the gains and the limits: the M4 winner was 18% more accurate than a naive benchmark (OWA 0.821 against 1.000), and on ForecastBench's November 2026 leaderboard the best AI entries are level with the superforecaster median (Brier Index 69.5% against 68.2%, overlapping confidence intervals). Precogs Chat explains how the models work, how to read their uncertainty and how to score them, with a free calculator.

Predictions are inputs, not verdicts. These guides show how forecasting models work, where they hold up, and how to check one against outcomes before you rely on it.

Forecast score calculatorMAE, RMSE, MAPE, interval coverage and the Brier score from your own numbers. How forecasting worksBenchmarks, statistical models, combinations, machine learning and what M4 found. Reading uncertaintyWhat an 80% interval promises and how to check calibration. Where forecasts are reliableAccuracy by data level, frequency and question type, from M4, M5 and ForecastBench. The review loopLog, score, compare with a naive forecast and retire what never beats it.

What the public benchmarks show

Benchmark What was forecast Best entry Simple reference Source
M4 Competition (2018) 100,000 time series, yearly to hourly Smyl (hybrid): OWA 0.821, sMAPE 11.37 Naive2: OWA 1.000, sMAPE 13.56; Comb (average of three exponential smoothing methods): OWA 0.898 Official M4 scores
M5 Accuracy (2020) 42,840 Walmart unit sales series, 28 days ahead YJ_STU: WRMSSE 0.520 Best benchmark (ES_bu, exponential smoothing): 0.671; seasonal naive: 0.847 Official M5 scores
ForecastBench paper (2024) 200 questions about future events Superforecaster median: Brier 0.096 Public median: 0.121; best LLM (Claude 3.5 Sonnet): 0.122 Karger et al.
ForecastBench tournament (updated 11 November 2026) Rolling questions about future events Top AI entry: Brier Index 69.5% (95% CI 68.1 to 71.1) Superforecaster median: 68.2% (66.8 to 69.8); public median: 63.0%; always 50%: 50.0% ForecastBench leaderboard

All checked 13 November 2026. Lower is better for OWA, sMAPE, WRMSSE and the Brier score; higher is better for the Brier Index, which ForecastBench scales so that 100% is perfect and 50% is always forecasting 50%. OWA is the M4's overall weighted average of sMAPE and MASE relative to Naive2, so 0.821 means about 18% better than Naive2. More results, by data level and frequency: where forecasts are reliable.

Score a forecast in your browser

The forecast score calculator takes actuals and forecasts, or probabilities and outcomes, and returns MAE, RMSE, MAPE, bias, a naive-forecast comparison, interval coverage with the Winkler score and the Brier score. The formulas and the cases where each metric misleads are listed under the tool.