On 11 January 2023, Émile Servan-Schreiber asked ChatGPT for a biography of himself and posted an answer he said was mostly false. It mixed his name with other people’s careers and attributed a book to him that he had never heard of. The failure was concrete: a fluent paragraph had assembled incompatible pieces of information into an authoritative-sounding identity.
One error can be checked independently. The answer credited him with The Laws of Disruption. The publisher, Basic Books, identifies Larry Downes as the author and dates the book to October 2009. Servan-Schreiber also rejected the chatbot’s claim that he had founded and edited Le Nouvel Observateur. His objection concerned claims about his identity, rather than the grammar or style of the response.
A personal error, rather than an accuracy benchmark
Servan-Schreiber speculated that limited material about him in the model’s training data had led it to blend nearby names and facts. That was his explanation, not an inspection of the model’s internal process. His estimate that roughly 80% of the answer was false was likewise an informal assessment of this response, not a measured error rate for ChatGPT.
The problem was already acknowledged at launch. OpenAI’s 30 November 2022 introduction warned that ChatGPT could produce plausible but incorrect answers, and that responses could vary with prompt wording. That contemporary documentation supports treating the anecdote as an example of a recognized limitation. Servan-Schreiber expressed a preference for Wikipedia’s collective approach, linking his personal experience to his interest in how many contributors can check information.
The February contrast: answers that can be scored
On 23 February 2023, he turned from biography to forecasting. He reported an analysis of 875,735 probability forecasts covering 2,535 possible outcomes across 816 questions over 8.5 years of Hypermind activity. Those are the figures in his post. The statistical record is covered separately in the journal’s Hypermind calibration history. His contrast raises a practical question: how can a confident answer be checked, whether it concerns a person’s past or an event in the future?
He argued that the ChatGPT of that period could write impressively yet declined requests to forecast elections, battles or market crashes. His broader statements about AI reasoning and forecasting were his contemporary judgement. They help explain why he valued probabilities tied to questions and outcomes, while his biography example required a different check: consulting the publisher’s factual record. One response supplies no controlled comparison of forecasting performance.
