"Trump Leads Clinton by 1 Point in New Poll as Enthusiasm Declines" blares the headline on an ABC News story about next week's presidential election. That may well be true. But it is also data malpractice.
Why? Two reasons.
1) They are deliberately measuring - and emphasizing - something irrelevant. Buried after 10 paragraphs of text and 2 large graphics: "In blue states (with 278 electoral votes), Clinton leads 54-37 percent." Um, one candidate has a large lead in states with 278 electoral votes, and only 270 are needed to win.
Nowhere in this ABC Web story did it even mention that Hillary Clinton is comfortably ahead in states with enough electoral votes to win the election. National political opinion may or may not be close. It might indeed be interesting to look at national opinion versus how the Electoral College is shaping up. But the popular vote doesn't matter when it comes to who wins (just ask not-former-President Al Gore).
This is just the most egregious example of "the presidential race is tightening!" media coverage of national political-opinion numbers that misleads people into thinking the contest is closer than it is. It's a bias of some - but not all - reporters, editors and producers who want the election to be a nailbiter because a close race is a lot more interesting than one that looks to be mostly decided. Who wants to say "this subject I've been writing about incessantly for two years actually isn't exciting anymore?" Admitting that the odds of one candidate winning the election are pretty substantial risks a decline viewers and Web clicks (not to mention charges of partisanship from the candidate who's behind).
But measuring popular vote instead of state-by-state electoral college polls is sort of like of following the World Series by tracking how many runs each team scores overall. What matters is number of games won. You can also talk about batting averages and ERAs and, yes, runs scored. But the main news is who wins. In a playoff series, one team's 5-2 win does not outweigh another team's two victories of 2-1 and 1-0.
2) Poll results -- even ones with excellent methodology -- are likely to bounce around. That's partly because of polling margin of error but also confidence intervals. Usually, polls report margins of error at 95% confidence intervals (or sometimes 90%) -- but the confidence interval is rarely included in news reports. What a 95% confidence interval means, though, is that even with correct methodology, 1 time out of 20, the actual value of candidates' support is going to be outside the poll's margin of error. (And this doesn't even account for the added complexity of trying to model which people will actually turn out and vote as opposed to those who have opinions but stay home.)
As Sam Wang at the Princeton Election Consortium explained in his post, Why did the polls seem so variable this week: "The basic answer is that there were a lot of them. Outliers are an inevitable consequence. . . . this amount of variation is totally natural. It’s kind of like watching dogs in the park. If you wait for ten of them, you’re more likely to see a big or small one. But what we want here is the median dog. The more polls there are, the wider the range of outcomes that you’ll see." (emphasis mine). A week before the election, there are likely going to be lots of polls released.
Experts who are serious about trying to make sense of political polls advise that it's best to look at poll aggregates. That's how you don't give outliers too much weight. Nate Cohn at the New York Times, whose coverage of political data has been outstanding, had some excellent advice.
"You’ve probably heard that it’s always best to focus on the average of polls. The poll results that often get the most attention are outliers — they get attention because they’re shocking, not because they’re representative," he wrote in his The Savvy Person'a Guide to Reading the Latest Polls. "But we also know that people often ignore that advice. They want to know the details of the newest poll, and how and why it might be different from the last. And to tell you the truth, I do, too. I read the details and methodology of almost every survey that is released."
If that's you and you want an informed way to look at the latest poll numbers and not just aggregates, read his Guide. But also remember that if the poll is just measuring national opinion and not Electoral College breakdowns, the results don't mean much -- no matter how accurate they are.