We are at that time of the election cycle where every close poll causes Democrats to have panic attacks. But it is important to understand the exact imperfections of polls rather than the myths about polls.
First, almost every polling company tries to be accurate. There are a couple of exceptions that are really propaganda companies that use slanted polls to push a political agenda, but most polling companies depend on having a reputation for accuracy.
Second, every poll has a margin of error. The margin of error is tied to sample size. To use a real world statistical example, we know that a coin flip will come up heads 50% of the time. But if you flip a coin twice, you will only get a 1-1 split 50% of the time. If you flip a coin 100 times, 95% of the time, you will get between 44 and 56 heads (a margin of error of 6%). If you flip a coin 1000 times, 95% of the time, you will get between 480 and 520 heads (a margin of error of 2%).
Third, that margin of error is based on random selection. But, in 2024, there is a nonrandom deselection. Given spam/phishing in calls, texts, and e-mails, a growing number of people will not answer a call, text, or e-mail from an unknown source. I know that I am likely to let calls go to voice mails if I do not recognize the number. Additionally, not every call is answered by a registered voter, and not every registered voter will vote.
Now, if there were no differences in preferences between people who are likely to vote and people who are not likely to vote and between people who respond to surveys or people who do not, this deselection would not matter. But we know from past experience that people are not interchangeable when it comes to surveys. Different demographic groups have different political beliefs. That does not mean that groups vote uniformly. They do not. But there are differences based on factors like race, gender, sexual orientation, education, union membership, religion (including whether a person frequently or infrequently attends services). As such, it is important to make sure that the respondents accurately reflect the voting population. Every pollster has their assumption (based in part on exit polls from the last election) of what the proper distribution of voters is. They will “weigh” responses (giving more weight to respondents from underrepresented groups and less weight to respondents from overrepresented groups) to adjust the raw data to reflect the total population.
More significantly, we do not know who will actually vote in 2024. The pollsters have questions that they use to identify who is likely to vote, but those questions are an imperfect tool for predicting who will vote. Very few pollsters release a high turnout, normal turnout, and low turnout result from the polls. And even if they did, not every group is going to have a high turnout or a low turnout. Figuring out who will win requires figuring out which groups are going to turnout (one reason why Get Out the Vote efforts matter).
Because of the inherent flaws in identifying who will vote and getting the balance of demographic groups correctly and random error, there will always be some errors. But because who turns out is not the same from election to election, those errors work in both ways. Some years, the actual votes are better for the Democrats than the polls. In other years, the actual votes are better for the Republicans than the polls. But, until the votes are cast, we do not know which way the polls will be off.
Some have concerns that, in both 2016 and 2020, Trump did better (nationally and in some states) than the polls suggested. But polling companies are aware of this issue and, because they want a reputation for accuracy, they have made adjustments. But polling companies do not publicly announce how they screen for likely voters or how they weigh the responses. Thus, we do not know which ones have things right.
Despite the assertion of errors in 2016 and 2020, the errors were mostly within the expected margin of error. A 3.5% margin of error applies to the vote for each candidate. If Candidate Ais getting 48% in a poll, that result merely indicates that Candidate A will get between 45% and 51% of the vote. If the Candidate B is getting 47% in that poll, that merely indicates that Candidate B will get between 44% and 50% of the vote. So that poll could mean that Candidate A will win 51-44 or that Candidate B will win 50-45. And, most polls include some undecided vote. We do not know who the undecided voter will vote for. While undecided voters will not all split the same way on election day, it is not unusual for there to be a significant split in late deciders with one candidate getting a big advantage.
Some of the polls in the field are “internal” polls commissioned by a candidate or groups that support a candidate. Internal polls are not necessarily inaccurate. Candidates and groups need accurate internal polls to tell them what messages are working and where to put more effort. But some polls are “push” polls to see if a message might move voters. While the people responding to the poll have gotten that message, that message has not necessarily penetrated with voters generally. And when a campaign or advocacy group releases a poll, there is a reason for that release. Both trailing and leading candidate have incentives for releasing outlier polls (showing a close race or even adverse numbers to spur fundraising or push higher turnout). And we do not know what the unreleased polls are showing.
Finally, that random sampling error should, on occasion result in outlier polls. Sometimes, an outlier poll is actually getting it correct with the mainstream polls being off. But, more often, the outlier is the once that is wrong simply as it is more likely that one poll is off than nine polls being off. And, even without outliers, a shift of 1% in a poll could just be the result of the random sampling rather than a true change in perspective.
So what should savvy readers be looking for when seeing daily polls? First, look for the consensus of the polls. Most of the time, if one candidate is ahead (even by narrow margins) in every poll, that candidate will be the winner. Second, compare the current poll to the last poll by that company. You are looking for the change in that poll. If multiple polling companies are showing similar changes (i.e. a narrowing or widening of the race by 1-2%), that is probably a real change. Third, recognize that polls tend to have leans. Over multiple polls, it will become clear that company A’s model assumes more Republican voters than company B’s model. As noted above, we do not know which model is correct, but knowing the tendency of that model is useful for determining how significant a new poll result is. If a company has uniformly been showing better results for Donal Trump than other companies, a new poll from that company showing that Donald Trump narrowly trails in a state is less significant than the same numbers from a company that has been showing better results for Kamala Harris.
In short, Democrats have reason to worry, but not reasons to panic. This country is divided, and we need to get every Democrat to vote in this election. The races will be close — for President, for Senate, for the House, for state and local governments. But anybody who says that the polls indicate that the race is over is pushing an agenda rather than speaking honestly.
