Methodology

Election Polling

One of the most prominent applications of survey research is election polling. In election years, much of the polling by the Pew Research Center for the People & the Press focuses on people's engagement in the election, opinions about the candidates, the campaign and voter preferences. Even in the so-called "off years," many of our polls include questions about party identification, past voting behavior or voter reactions to events.

Pre-election polling provides one of the few times when pollsters can assess the validity of their work by measuring how well their polls match election outcomes. But, polls designed to measure voter intentions serve up some special challenges. How do you identify which respondents will actually vote? Are respondents honest when they tell us for whom they intend to vote? How will undecided voters make their final decisions?

Although election polls attract a great deal of attention for their ability to predict the outcome of elections, their most important function is to help journalists and citizens understand the meaning of the campaign and the election. Polls help to explain, among other things, what issues are important, how candidate qualities may affect voters' decisions, and how much support there is for particular policy changes.

Identifying likely voters

One of the most difficult aspects of conducting election polls is determining whether a respondent will actually vote in the election. More respondents say they intend to vote than actually will cast a ballot. As a consequence, pollsters do not rely solely upon a respondent's stated intention when classifying a person as likely to vote or not. Most pollsters use a combination of questions that measure intention to vote, interest in the campaign and past voting behavior. Different pollsters use different sets of questions to help identify likely voters.

The Pew Research Center's likely voter questions from 2008 can be found in Pew Research Election Questions in red type. We use nine questions to assign each respondent a score on the likely voter scale in our final pre-election poll. Earlier in the campaign, we use a somewhat shorter version of the scale to identify likely voters. The following analysis provides more extensive detail about likely voters in the 2008 presidential election and the scale used to determine who was most likely to vote.

The Pew Research Center's likely voter scale is very similar to the method developed by election polling pioneer Paul Perry at the Gallup Organization and has been used successfully in many elections. To help assess which questions or combination of questions from the scale were most accurate in predicting voting, we conducted an experiment in a closely contested mayoral race in Philadelphia in 1999. This report describes that experiment and what we learned from it. Along with the academic paper, which is linked to within the report, it also provides more detail about how the likely voter scale is created and used.

These reports contain the results of the Pew Research Center's final presidential election forecast polls:

Back to top

Determining voter preference

Determining voter preference among the candidates running for office would appear to be a relatively simple task: just ask them who they are going to vote for on Election Day. In fact, differences in how this question is asked and where it is placed in the questionnaire can affect the results. While most voters have usually made up their minds and are not likely to be affected by how the question is posed, many people have given less thought to the campaign or are genuinely ambivalent about the choices. For these voters, certain features of the question can make a difference.

The questions in Pew Research Election Questions in blue type were used by the Pew Research Center for the People & the Press in its final poll of the 2008 presidential election to measure voter preference. The particular features of these questions reflect several choices:

  • Both the presidential and vice presidential candidates are included in the questions.
  • The party affiliation of each ticket is mentioned explicitly.
  • In states where Ralph Nader and/or Bob Barr were on the ballot, they and their running mates are included in the choices read to respondents. (Other minor party candidates were not named in the questionnaire.)
  • The order of presentation of the Democratic and Republican tickets is rotated so that some respondents hear the Democratic ticket first and others hear the Republican ticket first. The Nader and Barr tickets always follow the major party candidates. In states where both are on the ballot; they are also rotated.

These features are an effort to make the presentation of the options as similar as possible to what voters would actually experience when casting their ballots. Because Nader and Barr were not on the ballot in all states, respondents were asked whether they favored these candidates only in the states where they were listed as an option. And the rotation of the order of the two major party tickets reflects the fact that the order of the ballot may vary in different locations. However, this effort is not perfect, since not all states mention the party affiliation of the ticket and not all states feature a rotation or random selection of the ballot order. In addition, there are often other candidates on the ballot. The Pew Research Center and most other national polling organizations make a judgment as to which third-party tickets should be included in their survey questions.

Pollsters face an even more difficult challenge in primary elections, where the number of candidates is often very large and many include names that may be unfamiliar to voters. Long lists of candidates can create difficulties in a telephone survey and the effect of a candidate's position on the ballot can be more consequential than in a general election contest where fewer candidates are listed. In general, our practice in primary elections is to read all of the candidates that remain in the race, randomizing the order of the names.

Two other choices in the Pew Research Center's election questions are important to note:

  • The trial heat questions are asked very early in the questionnaire, prior to any other substantive questions about politics other than voter registration, political engagement and past voting history. This is done to avoid the possibility of affecting the voter's choice by raising considerations such as issues, candidate personalities, or other factors. While all of these may ultimately be relevant to the voter's choice, there is no guarantee that the things we mention will be the ones most important when a voter finally makes a choice among the candidates. Thus, it is important to make the choice as "clean" as possible.
  • Some voters will not express an initial choice so they are asked a follow-up "leaner" question (Q3a, Q3a_1, Q3a_2, or Q3a_3 in Pew Research Election Questions) in an effort to obtain a preference. People who lean toward one of the candidates are typically included in the tabulation of voter preference.

The remaining questions in the series shown in Pew Research Election Questions are used to gauge strength of support (Q3b) and certainty of support (Q5 and Q6). We also sometimes distinguish between "positive" and "negative" voting by asking whether someone's vote is more a vote for their selected candidate or against the other candidate.

The Pew Research Center also reports the size of the so-called "swing vote" -- defined as voters who are either undecided, only leaning to a candidate, or who say they might change their mind before Election Day. In addition to discussing the swing vote in many of our election reports, the following report describes a more extensive analysis, conducted in the late stages of the 2004 campaign, of the size of the swing vote and how swing voters who were identified in earlier surveys responded when re-contacted in mid-October:

One final issue in determining voter preference is the question of whether respondents will always answer honestly when asked for their choice in an election. For the most part this has not proven to be a problem, as most election polling has been very accurate. However, a small percentage of people - typically less than 5% - will refuse to answer the vote choice question.

A pattern of polling errors during the 1980s and 1990s in elections involving African American candidates raised the question of whether some people are reluctant to say that they are voting against a black candidate. Alternatively, there is the possibility that members of some demographic groups that are more likely to be racially conservative are also disproportionately likely to refuse to participate in surveys. If so, this could potentially produce a bias in the poll's estimate of the outcome of the election. Regardless of what caused it, polls in many of these elections tended to understate the level of support for the white candidate.

This phenomenon is sometimes called the "Bradley effect" because it was first observed in the 1982 California gubernatorial election between Tom Bradley, a black Democrat, and George Deukmejian, a white Republican. The Pew Research Center has examined the question of whether polling in elections continues to understate support for white candidates when they are running against black candidates. While the pattern was clear in the 1980s and earlier in the 1990s, more recent elections in 2006 showed little sign of the so-called "Bradley effect."

Concerns about the Bradley effect had obvious relevance for the 2008 presidential election, both in the Democratic primaries and in the general election contest between Barack Obama and John McCain. There was, however, no evidence of systematic polling errors consistent with the Bradley effect in either the primaries or the general election.

Back to top

Gauging the accuracy of election polls

Although polls have occasionally failed to predict who will win an election (most recently in the 2008 Democratic primary election in New Hampshire), polling's track record is actually very good. The National Council for Public Polls has conducted analysis of presidential election polling accuracy from 1936 to the present and provides reports summarizing these results on their website.

The good track record of final pre-election polls does not mean that all pre-election polls are reliable. Polls conducted early in an election season should be taken as snapshots in time, and obviously cannot capture the impact of the campaign and events to come. This commentary examines presidential election polls conducted well in advance of the election and attempts to gauge how predictive they are:

These commentaries provide a few tips to help in reading polls and deciding how much weight to give them:

National polls sometimes attempt to gauge how voters will vote in elections for the U.S. House of Representatives. Of course, there is no national election for the House; instead there are elections in each of the 435 congressional districts. But pollsters have found that the so-called "generic ballot test," which asks whether respondents intend to vote for the Republican or the Democratic candidate in their local race for the House, can provide an accurate estimate of the vote on which projections about party gains and losses in seats can be based. The following commentaries and analyses illustrate the use of the generic ballot test and some of the issues involved in using it:

Back to top

Additional topics on election polling

A number of Pew Research reports and commentaries provide additional examples of how election polls are conducted and used. Reports in the following section illustrate in-depth pre-election polling where the goal is to analyze and classify voters and to track changes in fundamental political attitudes and values over time.

In-depth voter analysis

Presidential election primary state polling

Primary elections present special challenges. These reports are based on polls in early primary states or describe the issues involved in polling these contests.

Voter reactions following presidential elections

The Pew Research Center has regularly polled voters after the election to gather reactions to the campaign and the outcome of the election. Most of these involve re-interviews with respondents from pre-election polls. These reports describe the findings from these surveys.

Back to top