As the tension — and exhaustion — levels rise in these last few days before the US election, opinion polls range from it’s too close to call a Trump/Harris victory to a Trump/Harris debacle.
The nerves and anxiety (and not just in America) are not helped by the huge number of polls, which is quite different from Australia where we traditionally focus on just two or three regular ones. Another difference is the high number of “swing state” polls. Whole states are easier to poll accurately than our federal electorates which is why we see so few published polls of marginal seats in Australia. (Pennsylvania, for example, has a population of 13 million compared with our federal electorates which typically number in the tens of thousands of enrolled voters.)
Generally it’s easier to have political polling in Australia because unlike the US we have compulsory voting. America’s pollsters have the tricky task of assessing which survey respondents will vote come election day and which will not. This is one part of the reason why Americans now mostly focus on poll averages. No single poll gets much attention, unlike in Australia.
The other part of the reason is the increasing difficulty of getting representative samples — in every country. Just 20 years ago, random calls on landlines provided satisfactory samples that could be enhanced to a high degree of reliability with modest weighting. Now a range of techniques including mobile phones, managed panels of people who do surveys for a small reward, and several social media channels are blended in different combinations, and some are better than others.
The upshot is that samples are less representative than they used to be, which means the stated (mathematical) margins of error (usually around +/3%) underestimate the actual variability from the correct vote numbers, which are unknown until election day.
Trumping to conclusions
Polls in America have been heavily criticised for significantly underestimating Trump’s vote in 2016 and 2020. Pollsters have said they’ve taken steps to correct this. We’ll see. At the moment, with polls suggesting the election is on a knife edge, if they are correct anyone could win. If the pollsters again underestimate Trump’s vote, he’ll win, and if they’ve over-allowed for past errors, Harris will win.
One remarkable feature of the current US campaign has been the rise of partisan polling. Partisan polling is typically funded by right-leaning groups. The polling produces results which are way out of kilter with established polls, showing huge support — up to 75% to the Trump camp. They are never factored into calculations of polling averages, such as are done by reputable news organisations.
But the aim of partisan polling is not to be accurate. It is to put into play via social media the idea that Trump is set for a landslide win. As The New York Times reports this is part and parcel of a strategy to pump up the “rigged election” conspiracy should Harris win. According to the NYT, in the final stretch of the 2020 campaign, Republican-aligned pollsters released 15 presidential election polls of swing states. In the same period this year, they have released 37, according to the paper’s analysis. Of those 37, all but seven had Trump in the lead.
One “new” technique US pollsters have been using is weighting by vote recall, which is simply based on a question such as: “In the last federal election in 2020 did you vote for Joe Biden or Donald Trump, or did you not vote?” This allows pollsters to see if one side is over-represented in their samples. They simply weight the figures to reflect the correct distribution of actual votes in the previous election. So if one is short of Trump voters from 2020, they are weighted to their actual (and known) level. It doesn’t matter if some of them now support the other side; these form part of a vital group known as switchers.
Americans are calling “recall weighting” new which is surprising us because Australia has used it for 30 years. But it has its pitfalls and could be described as more art than science. The main problem is that recall invariably embodies errors. The human memory is not one of nature’s better creations. Or sometimes respondents choose not to admit who they voted for last time, and lie.
An infamous example of this in Australia was after the 1998 Queensland election in which One Nation won 23% of the vote and 11 seats. The “two Davids” scandal and defections soon reduced its number of seats to four. Vote recall for One Nation in Queensland collapsed after this. Obviously, if the One Nation recall in subsequent polls had been weighted back to 23%, its set of 1998 voters would be greatly overweighted in the final sample.
It’s this unreliability that causes recall not to be used for most polls which, more scientifically, weight to ABS demographic distributions, notably age, sex, location and education.
Swings and roundabouts
But sometimes there are clear indicators the sample is distorted towards one side or the other. For instance, after demographic weighting a sample might indicate a sizeable swing away from a government despite high levels of satisfaction with it and its leader, and within the sample more respondents have moved to it than away from it. In such an instance, unless first-time voters overwhelmingly favour the other side, the chance is high that opposition voters have been over-sampled. Should the pollster then apply a full recall weight, or maybe tone it down with a half recall weight? It’s arbitrary.
Recall weighting is best when several surveys are done within a limited time among the same population, as is occurring now in swing states such as Pennsylvania. The recall errors average out and become stable. But we shall have to wait and see if US pollsters got all this right, given their lack of experience with this awkward variable.
John Utting and Bruce Dier have worked together on political and campaign polling for 26 years.
0 Comments