Press X to Doubt Eval: I told 14 AI models it's September 2026 and showed them 20 things that actually happened this year without websearch. On average they gave reality a 36% chance.

An evaluation tested 14 AI models by telling them it was September 2026 and presenting 20 real events from that year, without web search access. The models were asked to estimate the odds each event had already occurred and the month it happened. On average, the AI models assigned only a 36% chance to these actual events. The experiment highlighted a trend of AI models struggling to predict their own progress, with the caveat that this was a fun, informal test and not a formal paper.

This informal evaluation uniquely tests AI models' ability to predict future events by presenting them with 20 real occurrences from a simulated future date, unlike typical evaluations focused on current knowledge.