US election: why were the 2020 polls so wide of the mark again?

Joe Biden has it – maybe. As things stand, he has secured Michigan and Wisconsin, and legal challenges and recounts will almost certainly do nothing to change this. This puts him securely on 253 Electoral College votes, and needing 17 more to win. In play for him is Pennsylvania with 20 votes which gives him an automatic win. Victory in any two of these three states – Arizona (11), Nevada (6), and Georgia (16) – would also do.

As things stand Biden looks likely to pick up Pennsylvania. Trump’s initial lead here, which looked so insurmountable, has been eaten away by the counting of overwhelmingly Democratic mail-in ballots. Georgia has also seen its margins tighten for the same reasons. An increasing number of pundits are predicting potential Biden victories in both these states.

Nevada also looks positive for Biden. Early results gave him a lead, but only a narrow one with about 1/3 of votes still to be counted. Howevet, aince then his lead has extended.

Still, in Arizona things could swing the other way. The count is ongoing and has begun tightening in Trump’s favour, but it is unclear if it will be by enough. There’s a certain irony to the fact that this state was called early for Biden by Fox News of all outlets, infuriating Trump and playing a vital role in convincing many that Biden was still holding the edge despite grim news elsewhere.

Trump is still fighting tooth and nail to try and grasp some advantage. Legal cases and recounts are being prepped and funds are being desperately gathered by a cash-strapped Trump campaign to pursue them, despite there being little chance that these will affect their results.

Meanwhile, standing outside the polling stations, Trump’s supporters – egged on by his inflammatory rhetoric about the election being stolen – are trying to interfere in his favour. This creates the slightly through the looking glass scenario in which armed protestors are angrily chanting “Stop the count!” in Georgia, while their compatriots in Arizona are yelling: “Count the vote!”.

All of this begs the question – how did it get so damn close?

The first thing to note is that the polls got it wrong again, horribly. The margins of error were even bigger than in 2016 – even if they seem to have called the ultimate winner correctly. We had been told that the pollsters had fixed the issues in the 2016 polls – in particular they were polling more non-college educated voters.

Well, whatever they did it doesn’t seem to have worked. Maybe in the future we shall turn to more reliable methods like astrology, crystal balls, and reading the livers of ritually-sacrificed goats.

As for what scrambled the polls so badly, we probably won’t know for a while – but there are some preliminary indications.

The first key factor seems to be the massive increase in turnout this year. Polls are “weighted” (mathematically tweaked) to try to improve accuracy by better reflecting the expected composition of voters. One thing that can mess with this is changes in turnout – and preliminary estimates suggest a turnout rate 66.9%, with 160 million voters participating in this year’s election. This would be the highest percentage since 1900, utterly dwarfing the 2016 turnout of 59.2%, or nearly 137 million voters.

Many of these 30 million extra voters are likely to have been first-time voters – 13% of the total 2020 electorate this according to NYTimes exit polls. Others probably had not turned out for a long time. It seems pollsters, while expecting increased turnout, just flat out missed whatever groups were taking the unusual step of turning out this year.

Usually its assumed that higher turnout helps the Democrats – as they do well among low turnout groups like ethnic minorities and the young. However, Trump proved this year that the Republicans can play the turnout game as well.

This brings us to the second key point. Donald Trump managed to find new voters. Most startlingly exit polls, while preliminary at moment, suggest that many – but not all of these votes – came from the black and Latino voters. Trump won 12% of black voters and 32% of Latinos.

The fact that Trump of all people has been able to do this – despite Democrats decrying him as racist as they ramped up their identity-based messaging – shatters a lot of liberal pieties. It also takes some explaining – and can be read in a number of ways.

Prior to the election, I discussed Trump’s apparent inroads among minorities with US political scientist Omar Wasow. One theory he suggested was that this was a function of growing polarisation along gender lines – with men being more likely to appreciate Trump’s more assertive style.

There is perhaps something to this, but it is clearly not the whole story. Exit polls suggest that while Trump made his biggest gains among minority voters with men, they also show Trump increasing his share of the vote with black and Latino women.

Other factors are clearly at play. One possibility is increased polarisation along educational lines overriding racial polarisation in some cases – though this actually seems to have been reduced overall in this election, given that Biden made some gains among non-college educated white voters when compared to Hillary Clinton in 2016.

On the flipside, one could also put this down to Democratic failure. Biden’s weakness with many Latino voters was apparent in the Democratic primaries. During his presidential campaign, he was criticised for not reaching out more to this demographic. As for black voters, one mid-level Democratic organiser who worked the Democratic campaign in Georgia admitted to me that internal polls had suggested Trump was going outperform expectations among black voters – but, he said, activists on the ground had dismissed the findings.

The case for Democrats being complacent and assuming that Trump would be naturally repulsive to these groups grows if one looks at previous Republican performances. Since Reagan, Republicans have regularly won more or less 9-12% of the black vote. The only exception to this was when this share collapsed as black voters flocked to Obama. In 2016, Trump got this back to 8%. As for Latinos, the Republican share of this groups votes has fluctuated in 20-30% region, with a high of 44% in 2004.

In this light of all this, Trump is slightly above the norm of Republican performance. Crucially, he also built on a base vote in 2016.

The night also shows that the concept of “Latino” is just not a very useful as a catch-all ethnic category. Biden’s sharp underperformance in Florida seems to be at least in part due to Cuban-Americans in Miami-Dade swinging sharply for Trump. This group has long been Republican leaning and Trump bombarded the community with ads tying Biden to socialism and communism – something that resonates in a community which originated as refugees from Castro’s revolutionary Cuba.

Biden also seems to have underperformed dramatically among “Latinos” in Texas. However, he seems to have done well with other “Latino” voters across the country, including in some key swing states such as Arizona and in the Midwest. One potential factor at play is that Latinos in Texas tend to be third and fourth generation migrants, while others across the country have often arrived more recently. A rural-urban divide has also been floated as a possible explanation for differences among such voters.

Finally, we have the electoral college. As in 2016, without this strange system, which does slant Republican, Trump’s election chances would be dead and buried. However, the fact that this the college favours states and demographics that lean Republican has kept Trump in the game.

Of all the factors helping Trump, this is perhaps the least surprising – it has been observed since the last election – but the most worrying. High turnout is a sign of electoral vitality. Republican inroads among non-white voters could help reduce dangerous racial tensions. However, a system that – with only a mild upset – might for the third time in five elections hand victory to the Republicans despite them having only won with a minority of the popular vote is a sign of systemic sclerosis.