All of the experts thought that Obama would win the New Hampshire primary. Perhaps more significantly, the prices at Intrade, the political prediction market, suggested that Obama was overwhelmingly likely to win. Many of us have been quite excited about the potential of prediction markets, in which people "bet" on political (and many other) outcomes. Such markets have a terrific track record. And yet Clinton was a huge underdog in New Hampshire. Should we conclude that the prediction markets are unreliable after all? That nobody knows anything? That polls themselves mean nothing?
The answers are no, no, no. The problem is that as in so many domains, a single, unlikely event (here, the stunning surprise in NH) is leading to a ridiculous generalization about the world.
More particularly: Intrade had Clinton at about 8% likely to win, and 8% of the time, 8% chances should come through. (As in horse races, so on Intrade.) It would be a great surprise if 8% chances come though 0% or even 2% of the time. But on prediction markets, prices have been close to probabilities. Events that are priced to be 90% likely to happen occur about 90% of the time, events that are priced to be 70% likely to happen occur about 70% of the time, and so on.
The current skepticism about prediction markets is being driven by the well-known availability heuristic, in accordance with which people assess probability by asking whether events come easily to mind, or are cognitively "available." The success of the long-shot in New Hampshire is now very available indeed. In fact we are witnessing an availability cascade, in which the single incident spreads rapidly from one person to another, leading to the (false) belief that the prediction markets are quite unreliable.
There is a more general point. Some people are now doubting not only the prediction markets but also the polls, saying that no one knows anything, and that anything is as likely as anything else. Don't believe it. Of course no one has a crystal ball, but the polls are generally pretty good -- and if you want to have a sense of the probabilities, you'd probably do best to consult Intrade.
--Cass Sunstein