Public Opinion Polls

What would you say are your chances of being polled?

  1. Very likely.
  2. Somewhat likely.
  3. Slim to none.
  4. No opinion.

The fact is, if you have a telephone and the poll is done correctly, your chances are about as good as anybody else’s. But if you’re like most people, you wonder why incessant political polls claim to report the views of “all Americans” or “likely voters” when nobody asked you.

Election-related polls can be especially controversial due to the widespread belief that they fail to reach certain kinds of people and can skew the outcome of a race.

Betty H. Zisk, a political science professor at Boston University, every semester explains the mechanics of public opinion surveying to her students and gets skepticism–nay, downright cynicism – in return. “My students have zero respect for polling,” Zisk says. “They don’t believe a thing in the polls.”

Research has shown that question order and question wording can affect a survey’s reliability. But among Zisk’s students, as with the public at large, little arouses more suspicion than sampling–how pollsters choose people to be polled and why the few selected are said to represent the huge number who are not. Zisk, who has studied polling, says it is wise to hone in on that issue: “Without a good sample, the whole thing is worthless.”

The most important ingredient? Randomness.

“You have to try to [select participants] as randomly as possible, giving everyone in your target group–whether it’s the adult population or voters–an equal chance of getting chosen,” explains John Gorman, president of Opinion Dynamics, a Cambridge survey research firm now working for Attorney General Scott Harshbarger’s campaign for governor.

Fortunately for the pollsters, there’s no poring over phone books or random-number tables required. As you might expect, computers do the work for them. Today most professional pollsters simply buy random samples for about $1,500 per list from companies that specialize in sample creation, says John Della Volpe of Della Volpe & Associates, a polling firm in Concord.

For a typical general-election poll in Massachusetts, samples come from a computer-generated database of every possible phone number in the state. (For primary polls, it’s more common to start with a list of registered voters from the Secretary of State’s office.) Boiled down to the essentials, here’s how it works: The computer takes a randomly selected area code and exchange, then adds two digits in a complicated process intended to eliminate non-working numbers. The last two digits are chosen at random.

This procedure, called random-digit dialing, is not the most efficient sampling method. It tends to yield a lot of faxes, modems, and business lines–unless a special computer program is used to weed them out. But it’s the only way to get non-listed numbers in the sample.

Once election pollsters finally get someone on the phone, they have to ask several screening questions to make sure they are speaking with a likely voter. If not, it’s on to the next number on the list.

Assuming all goes well, the polling process ends when the final interview is finished. But sometimes, a few adjustments can be necessary–at least according to some pollsters. “Sometimes you have to violate some of these rules [of random samples] to do what you want to do,” says John Gorman of Opinion Dynamics.

For example, it’s more difficult to reach certain demographic groups at certain times of day, Gorman explains. So if two-thirds of the respondents turn out to be women, even though men make up half of the population, the results can be corrected through “weighting”–giving the under-represented group more weight in the final tally.

Betty Zisk calls it the “finagle factor,” and says it can cause problems. John Della Volpe agrees: “Lots of times when polls are wrong it has to do with weighting…. It’s typically the area where I think people make some mistakes…. As long as you do a good random-digit-dial survey, you shouldn’t need to weight.”

But what about the people who don’t own phones? This used to be more of a problem. Today almost all Americans–an estimated 95 percent of U.S. households–do have phones, according to Herbert Asher’s Polling and the Public. However, those without phones are more likely to be poor, less educated, and minorities.

And here’s something else to consider: An estimated 14 percent of households with phones have two or more different phone numbers, according to one sampling firm’s literature. That means no matter how random the sample, these people are at least a little more likely to be included. And logic says they are more likely to be wealthy.

As for the size of a typical statewide sample, most Massachusetts pollsters agree that contacting 400 people is plenty. How can so few represent so many? (The state does have more than three million registered voters.) For those who don’t revel in the technical, it probably has to be taken on faith.

As John Gorman puts it: “The explanation of why it works rests on a large body of statistical knowledge. There’s no short way to explain it and there’s no simple way to explain it. Go out and get a good elementary textbook on statistics.”

Meet the Author
Part of the answer has to do with the fact that a 400-person sample is big enough to yield an acceptable margin of error–usually plus or minus 5 percent. Doubling the sample size doesn’t much change the margin of error, so it’s not usually worth the time and expense of doing the extra interviews.

After many years in the business, veteran Democratic pollster Irwin “Tubby” Harrison says he’s used to the skepticism. “People will say, ‘How can you do that?’…’How come I’ve never been interviewed?'” Harrison says. “But I don’t pay attention to that…. Time after time, it gets you pretty close.”