Public Opinion Polling: A (Very) Brief Introduction For the Casual Observer

By John C. Davis

Earlier, I wrote about the results of a recent Talk Business and Politics/Hendrix College poll. Upon completing the piece, it occurred to me that it might be helpful if I shared some basic tips on how to evaluate polling results. While brief, and lacking attention to mathematical elements of scientific polling, the following is not unlike what I might say to my Introduction to American National Government students when giving them a very broad, brief description on polls and polling results.

1. Be aware that a poll, much like a picture from a camera, gives you a snapshot in time. While some public opinion topics remain relatively stable over long periods of time, others are more susceptible to change. In other words, it is best to assume that polling results have a limited shelf life.
2. Know the source of the poll. Often, we encounter news coverage on a poll conducted by someone other than the person or persons who conducted the survey. That is perfectly normal. However, if the source is not readily available in the article in which the survey is reported, that should raise a red flag.
3. Ask yourself, “what is the poll measuring?” For example, national polls reporting on the presidential race are interesting, exciting to some of us, and can be useful when comparing over time, but are not a reflection of the way in which we elect the president and are thereby particularly limited in their utility. Recall, in 2016, when people criticized polls and those who conducted them? Most often, the problem was not with the pollsters or with their methods, but with the way in which their results were reported and interpreted. A national poll might indicate one presidential candidate leading if, say, the election was held on that day and the votes were tallied by popular vote (in the case of 2016, the national polls often favored Clinton who did, in fact, win the popular vote). However, in the case of presidential selection, state polls—particularly in “toss up” or “battleground” states are most useful in evaluating where the candidates are stronger or weaker and better reflect the way in which Electoral College votes are awarded. Context matters.
4. Be aware of “push polls” conducted in such a way to coax a certain answer from a respondent. If the questions asked are not easily accessible in the story or at least in a link, this should set off alarm bells.
5. Review the sample size and, if available, demographic breakdown of those polled. Polling samples are usually determined by two factors: mathematical precision and cost. Properly conducted polls are expensive and get more costly as sample size increases. However, to reduce sampling error, mathematically, the larger the sample the better. Also, the demographics of those polled must also be taken into consideration. For example, if I were to conduct a poll to measure the support of an upcoming ballot question in Arkansas among likely voters, I would need a sample size of several hundred Arkansans, I would need to determine that they were likely voters (trickier than it may sound), and the collection of people I am polling would need to adequately represent the demographic breakdown of likely voters in the state, in order for me to expect my results are valid.
6. Review the margin of error. The margin of error, also known as the confidence interval, is the result of a mathematical equation based on the size of the sample. The margin of error should be reported with the polling results and, if nothing else, serve as a reminder that polls are estimates and are never intended to be 100% accurate 100% of the time.
7. Finally, consider the polling method and don’t waste your time on convenience polls (polls conducted among friends or others you follow on Facebook and Twitter come to mind). To be fair, technological advances and a significant drop in the use of landline phones are revolutionizing the way in which surveys are being conducted and I no longer write off any method of contact for obtaining a sample. However, I am leery of landline only polls (there is nothing wrong with having a landline phone, but how many people under the age of 35 do you know who own one?) and most online polling methods. I am becoming more comfortable with other means such as cell phone polling and text polling (when people won’t answer an unfamiliar number on their cell phone) if done by a reputable group.

This list is merely a basic introduction into public opinion polls intended to help you as we approach the 2020 general election campaign season. Have a more specific question? Email me.