Part I: Introduction

In a political race (particularly the Presidential race), there is a “horse race” aspect to it: people (particularly influentials and campaign donors) are interested in knowing who’s ahead, who’s fallen behind, who has a chance, who is a lost cause, and so forth. One of the means this narrative can be shaped is through the communication/dissemination of public opinion polls.

To appreciate the possible ways that this information can be “manipulated”, it’s important to understand what a poll is and isn’t. We had discussed this topic at some length in May. Quite simply, a poll is a representative sample of voters at a given point (or over an interval) in time. It is the “representative” part that is very subjective, and is arguably creating the narrative that President Obama has already won the election.

We would like to illustrate why these polls should be taken with a grain of salt by examining four “swing state”  polls that were released last week for Florida, Ohio, and Pennsylvania.

Part II: CBS/New York Times/Quinnipiac polls in Florida, Ohio, and Pennsylvania

The first “tip off” to us these polls were questionable were the Obama percentages: the last time a Democrat exceeded 53% of the vote in Ohio was in the 1964 LBJ/Democratic landslide. In Florida, no Democrat has received 53% since FDR’s 1944 re-election. And while President Obama carried Pennsylvania 54/44% in the 2008 elections, the last time a Democrat had a 12 point poll lead was in the 1964 LBJ/Democratic landslide, and before that, the 1936 FDR landslide.

After that, there were two things in the internals of the polls (i.e., the 20 pages of detail that people rarely read) that we found odd: (1) depending on the state polled, 64-72% of respondents have been paying “a lot” of attention to the election campaign – those numbers sounded suspiciously similar to voter turnouts (and suggests that a significant number of non voters were polled), and (2) among those paying “a lot” of attention to the race, a 9 point lead for President Obama became 4 points  in Florida, while Obama’s “lead” in Ohio went from 10 to 1 percentage point (curiously, the poll write-up did not mention what happened in Pennsylvania).

If you were to dig further, there are additional disclosures which enable us to further evaluate these polls: party enthusiasm was consistently higher for the Republicans, and there was an unusual choice of party breakdown of the respondents in each state (more on that later). In our view, enthusiasm is a fairly good tool to use to approximate party turnout in November. We found it noteworthy that the poll write-up neglected to mention the level of enthusiasm from Independents.

Finally, the party identification numbers are questionable in our mind, because they don’t approximate current voter registration statistics for any of the three states polled.

Therefore, we reconstructed these three polls by doing the following: (1) Rather than use the poll’s Democratic/Republican/Independent breakdown, we used the breakdown provided by the 2008 CNN exit polls (granted, this is skewed towards the Democrats, because 2008 was a disastrous year for the Republicans, but it’s still a respectable metric to use for this analysis), (2) using the Democratic/Republican enthusiasm to properly weight party turnout for each of the three states. Since data for Independents was not provided, we will assume that Independent enthusiasm is between that of the Democrats and Republicans.

When our recalculations are performed, we get rather different results:

Florida: 49-48% Obama

Ohio: 49-48% Obama

Pennsylvania: 49-46% Romney

Part III: Columbus Dispatch poll in Ohio

There is one more poll we would like to analyze: an Ohio poll reported in the Columbus Dispatch that showed President Obama with a 51-42% lead. While Republicans and Democrats were oversampled relative to the 2008 CNN exit poll, Independents were significantly under sampled. Furthermore, in the poll detail, respondents were asked who they supported for President, and those who voted in 2008 favored President Obama by a 55-43% margin. To put this in proper context, Obama actually carried Ohio in 2008 by 5 points (52-47%), so a poll showing an Obama margin more than double his 2008 showing is immediately questionable.

Once again, we used CNN’s 2008 exit polling numbers to approximate party preference, and applied Democratic/Republican enthusiasm (just like we did for the CBS/NYT/Quinnipiac polls) to get an adjusted poll number. Once that was done, the recalculated Ohio poll showed a much tighter race:

Ohio: 47-45% Obama

Part IV: Conclusion

We wish to close by reiterating ways to properly evaluate the numerous polls which are being conducted:

(1)   Consider the source – Who released/conducted the poll? Was it a political party or was it a TV station or paper? And was the polling firm one with known Democratic (or Republican) ties?

(2)   Who was polled – we are more interested in “likely voters” than “registered voters”, as voter files do not necessarily contain 100% likely and active votes. To illustrate, in Louisiana, 17% of registered voters have never cast a ballot, while another 11% last voted before the 2008 Presidential election. We have no reason to think it would be much different in other states, especially those states (like Minnesota and Wisconsin) that allow people to register and vote on the same day;

(3)   Trends – Most pollsters typically poll the same race multiple times. So when evaluating a poll release, it’s always good to know what the numbers were in a prior poll done by the same pollster to get the proper assessment for where the race is heading.