Dan Hopkins is an assistant professor in Georgetown University’s Government Department, and a contributor to Behind the Numbers.
Just before the holidays, Politico’s Roger Simon suggested that polls are “crystal balls” and “magic.”How else could interviews with a tiny fraction of voters yield numbers close to election results?
Just one instance from two weeks ago: Suffolk University’s final poll before the New Hampshire primary gave Mitt Romney 39.8 percent of the votes among respondents with preferences (calculated by ignoring undecided voters). He won 39.3 percent of the vote.
In response to Simon, others have been quick to explain that there is nothing particularly magical about random sampling. Even so, there is something about contemporary polls that is impressive and unexpected, if not a touch magical: the fact that overall, they remain accurate in forecasting general election outcomes.
This accuracy may be surprising. Response rates have tumbled. Consider the Survey of Consumer Attitudes, which has been asking Americans the same battery of questions about our economic perceptions for decades. In 1981, the response rate was just over 72 percent (gated); by 2008, it was a shade under 40 percent. Then factor in America’s changing demographics, which mean that as of 2007, approximately 10 million U.S. citizens did not speak English very well (pdf).Then there’s the well documented issue with cellphones.
Together, telephone polls face odds more daunting than did Tim Tebow.
And yet, over the past two decades, telephone polling has generally delivered accurate estimates of general election voting patterns. Take a look at the figure below. The gray dots represent the difference between the poll predictions and outcomes, based on a review of 1,402 publicly available, live-interviewer telephone surveys from gubernatorial and U.S. Senate elections from 1990 to 2010.
The black dots indicate the average difference for each year, with larger numbers indicating greater inaccuracy. Certainly, polls are sometimes widely off-the-mark — and these instances get a lot of attention — but the average error is under 3 percentage points. And there is little evidence that telephone polls have become less accurate over time. While “magic” might not be the right word, the accuracy of recent telephone polls in assessing voters’ preferred candidate was certainly not preordained, either.
More from the Post polling team
Sign up for Post polling e-mails
Follow Post polling on Twitter
Like Post Politics on Facebook
No comments:
Post a Comment