The New Republic

Oh, let me count the ways. Almost all of this, by the way, is lifted from Mark Bluemthnal's outstanding Exit Poll FAQ. For the long version, see over there.1. Exit polls have a much larger intrinsic margin for error than regular polls.
This is because of what are known as cluster sampling techniques. Exit
polls are not conducted at all precincts, but only at some fraction
thereof. Although these precincts are selected at random and are
supposed to be reflective of their states as a whole, this introduces
another opportunity for error to occur (say, for instance, that a
particular precinct has been canvassed especially heavily by one of the
campaigns). This makes the margins for error somewhere between 50-90%
higher than they would be for comparable telephone surveys.2. Exit polls have consistently overstated the Democratic share of the vote.
Many of you will recall this happening in 2004, when leaked exit polls
suggested that John Kerry would have a much better day than he actually
had. But this phenomenon was hardly unique to 2004. In 2000, for
instance, exit polls had Al Gore winning states like Alabama and
Georgia (!). If you go back and watch The War Room,
you'll find George Stephanopolous and James Carville gloating over exit
polls showing Bill Clinton winning states like Indiana and Texas, which
of course he did not win.3. Exit polls were particularly bad in this year's primaries. They overstated Barack Obama's performance by an average of about 7 points.4. Exit polls challenge the definition of a random sample.
Although the exit polls have theoretically established procedures to
collect a random sample -- essentially, having the interviewer approach
every nth person who leaves
the polling place -- in practice this is hard to execute at a busy
polling place, particularly when the pollster may be standing many
yards away from the polling place itself because of electioneering laws.5. Democrats may be more likely to participate in exit polls. Related to items #1 and #4 above, Scott Rasmussen has found
that Democrats supporters are more likely to agree to participate in
exit polls, probably because they are more enthusiastic about this
election.6. Exit polls may have problems calibrating results from early voting. Contrary to the conventional wisdom, exit polls will attempt account for people who voted before election day in most
(although not all) states by means of a random telephone sample of such
voters. However, this requires the polling firms to guess at the ratio
of early voters to regular ones, and sometimes they do not guess
correctly. In Florida in 2000, for instance, there was a significant
underestimation of the absentee vote, which that year was a
substantially Republican vote, leading to an overestimation of Al
Gore's share of the vote, and contributing to the infamous miscall of
the state.7. Exit polls may also miss late voters.
By "late" voters I mean persons who come to their polling place in the
last couple of hours of the day, after the exit polls are out of the
field. Although there is no clear consensus about which types of voters
tend to vote later rather than earlier, this adds another way in which
the sample may be nonrandom, particularly in precincts with long lines
or extended voting hours.8. "Leaked" exit poll results may not be the genuine article.
Sometimes, sources like Matt Drudge and Jim Geraghty have gotten their
hands on the actual exit polls collected by the network pools. At other
times, they may be reporting data from "first-wave" exit polls, which
contain extremely small sample sizes and are not calibrated for their
demographics. And at other places on the Internet (though likely not
from Gergahty and Drudge, who actually have reasonably good track
records), you may see numbers that are completely fabricated.9. A high-turnout election may make demographic weighting difficult.
Just as regular, telephone polls are having difficulty this cycle
estimating turnout demographics -- will younger voters and minorities
show up in greater numbers? -- the same challenges await exit
pollsters. Remember, an exit poll is not a definitive record of what happened at the polling place; it is at best a random sampling.10. You'll know the actual results soon enough anyway. Have patience, my friends, and consider yourselves lucky: in France, it is illegal to conduct a poll of any kind
within 48 hours of the election. But exit polls are really more trouble
than they're worth, at least as a predictive tool. An independent panel created by CNN in the wake of the Florida disaster in 2000 recommended that the network completely ignore exit polls when calling particular states. I suggest that you do the same.

--Nate Silver

share this article on facebook or twitter print this article