Back to Blog

Beyond historical data: the 2015 case for collective forecasting

3 min read
Navy cover headed “Collective forecasting beyond big data: the 2015 argument” beside a multicolored line chart titled “Evolution of likelihood of Republican Senate control,” comparing Hypermind with seven other forecasters.

On 22 June 2015, Émile Servan-Schreiber declared on LinkedIn that political polls were “brain dead” and that prediction markets would survive them. It was a deliberately provocative judgment. His longer essay, published on LinkedIn on 2 February 2015, made a more useful argument: forecasting can miss relevant information when it searches only what is already easy to measure.

The essay began with the familiar story of a person searching for lost keys beneath a streetlight because that is where the light is. Servan-Schreiber used it to question the assumption that a larger historical dataset necessarily contains the information needed for the next decision.

What happens when the past becomes a poor guide?

In his account, structured data suited computers, while implicit or ambiguous information could be pooled through human judgment. He argued that projections were particularly vulnerable when forecasting new products or markets undergoing disruption. These were claims about the strengths of collective forecasting in 2015, rather than a permanent boundary on what computers could process.

The business example concerned a Lumenogic collaboration with marketing researchers inside an unnamed Fortune 100 consumer packaged goods firm. The essay reported superior collective forecasts in 67% of cases, an average error reduction of approximately 15 percentage points, and an error-range reduction of over 40%. Those are Servan-Schreiber’s published figures; the original study behind that exact combination is unavailable.

An author’s university CV records a related 2014 conference paper by Mark Lang, Neeraj Bharadwaj and C. Anthony Di Benedetto. Their 2016 journal article describes a non-trading forecasting tournament and reports different results: 73% of the time, almost 34 percentage points of average error reduction, and over 40% for the error range. These later results support an employee-crowdsourcing comparison, while remaining distinct from the essay’s earlier figures.

A narrower, testable election comparison

On 14 November 2014, Servan-Schreiber had already announced that Hypermind had outpredicted the big-data models in the US midterms. Two days earlier, he welcomed PredictIt and said he was personally shorting Hillary Clinton. That trading opinion came without a probability, entry price or documented return.

The 2015 comparison by Servan-Schreiber and Pavel Atanasov assessed Senate control and five state races against seven statistical models and their average. It reports that Hypermind’s overall mean daily Brier score was 17%–46% lower than competing models’ scores. Lower scores indicate greater accuracy. The authors stress the small sample: six questions. Only the New York Times comparison reached p<.05; three other comparisons reached p<.1.

For Republican Senate control, the paper says Hypermind prices stabilized around 75% in June 2014 and stayed near that range until October. Republicans did take control: the Senate’s official history records 54 Republican seats for the 114th Congress, beginning in 2015.

The comparison supports a particular historical result. It does not settle every contest between markets and models, or demonstrate that unstructured information caused the advantage. That proposed explanation remains the authors’ interpretation.

The lasting question is practical: which information can a forecasting system collect, and how will its predictions be evaluated? Our guide to prediction-market probabilities explains how market prices express uncertainty. This earlier argument adds a reason to seek judgments beyond the dataset—and to keep the benchmark, question set and measured outcome visible when claiming success.

Evidence

Sources & further reading

Keep reading

Related articles

InsightsPrediction Markets Explained: How to Read Polymarket OddsForecastsHypermind’s 2016: ahead of the betting markets on Trump’s nomination, then one chance in four for Brexit and the White HouseInsightsWhen ChatGPT invented a biography: an early argument for accountable forecasts

Subscribe to The Forecasting Brief — forecasts & model notes, once a week