Polls are our primary tool for understanding and investing in public opinion, yet their complexity, high stakes, and competing uses breed distrust. Since 2016, the razor-thin margins in presidential elections and deepening partisan divides have heightened demands for polling accuracy – even as growing mistrust makes polling itself more difficult.

Maintaining public engagement with collective opinion is crucial. This series explores two questions: What sustains polls’ social legitimacy? And how do polls actually work, starting from first principles? By examining both the social contract behind polling and its methodology, we’ll understand not just what polls can and cannot do, but what they must do to serve democracy.

The three parts of the polling cycle

In sociology, institutions refer not only to complex organizations, but also to the shared rules different participants follow. Perhaps the simplest example of an institution is the handshake, which establishes the rules for introducing oneself to strangers. Just as handshakes formalize introductions, polling formalizes public opinion — establishing both the reality of diverse viewpoints and society’s expectation that leaders should respond to them.

The shared rules can be organized into a cycle with three main phases: production of polls, reporting on polls, and responding to polls. The response to polls is the phase in which public opinion changes. Arguably it lies outside of the polling cycle since it’s when the real world responds to the polling cycle, but it makes most sense to analyze polls in the context in which public opinion is actively changing, since that’s precisely when polls are needed.

Producing polls

The most important contributor to producing polls is the interviewee. Without willing participants, pollsters are worth nothing. When polling is more trusted, participants are more willing to participate, forming a virtuous cycle.

Pollsters are in control of some factors that contribute to whether a participant responds to a poll. They can’t control the general trust the participant has in the polling institution, but they can train their interviewers to sound trusthworthy, empathetic yet neutral, offer cash incentives that help to differentiate polls from spam, and so forth. The general trust in the polling institution depends on the perception of polls as accurate, fair, and non-elitist, all of which Donald Trump attacked in 2016.

Several organizations are involved in producing polls, including:

The pollsters themselves, who can be owned by or sponsored by media, universities, philanthropic grants, or political campaigns
Academia, which trains pollsters, advances methodology and researches political science more broadly
Industry organizations like AAORP which sets transparency and ethical standards
Even more broadly, telecommunication infrastructure that polls rely on to communicate, but itself can be less trusted via misinformation and spam actors

Reporting on polls

Polls are primarily reported on by way of forecasts, who fall into three main types: media organizations who often sponsor the poll they report, aggregators like 538 who achieve better forecasts by averaging polls, and betting markets who consume any information the bettor wants, certainly including polls or poll aggregations, to price in the odds of one candidate winning an election. In the modern day, individual pollsters are judged by their accuracy, which informs their weight in aggregators for upcoming elections, while the polling industry is perceived by the public through the lens of aggregators.

Disruption by betting markets

Betting markets have begun to displace poll aggregators as the primary election forecast used by the public. They offer distinct advantages in election forecasting through market-based probability assessment and the ability to incorporate events beyond voting, like legal challenges. However, they complement rather than replace polling. Firstly, like aggregators, they consume polls to produce a forecast.. It may take a research effort into its own right to determine exactly how much, but for one example fresh in memory, look to the impact Ann Seltzer’s Iowa poll had on manifold.market’s Iowa market. Secondly, they don’t produce some rich information besides a forecast that polls produce, like a demographic breakdown. Polls and markets will complement in each other in the near future.

Meanwhile, these markets are still young in their widespread use for election betting, and face their own challenges: we have little insight into who participates in these markets and drives the forecast, their association with gambling, the profit motive of the market owners, and feedback loops between election results and pricing (which polling forecasts also have). A counterproductive culture war casts bettors as right-wing charlatans versus pollsters as left-wing elitists. This false dichotomy undermines both tools’ value in understanding public opinion - the left shouldn’t ignore a capable forecasting methdology, nor should the right downplay the role of an accurate assessment of public opinion in Trump’s victories.

Responding to polls

The real world responds to polls in two ways: by trying to align with public opinion, and by trying to change public opinion. For politicians, this produces a challenge: they can’t win votes if they don’t align with anybody, but if they change their platform or identity too easily, they’ll be perceived as craven and chameleon-like.

What’s more interesting is when politicians or other actors (including celebrities, CEOs, activists, interest groups, and so forth), try to change public opinion. Paradoxically, we rarely speak of changing hearts and minds, even though changing hearts and minds is the lifeblood of… well, life. Instead, we talk about amplifying unheard voices (by teaching the dominant group about those voices), or uniting people against a perceived enemy (by forming a new group identity to replace the old ones), or how the Democrats have lost touch with the working class (and how we need to change the working class’s minds about the Democrats), or how people just need to learn who Kamala is (but we don’t want to say aloud what they think of her now).

Polls only exist to track change, and the only reason polls are hard to administer is because change is happening. If opinion is changing rapidly, it even produces the chance people’s opinions will change between the last poll and when they enter the voting booth. And if opinion about institutional trust is changing, polling’s methods will break down in addition to its results.

Donald Trump’s attack on polls

The best, recent example of a politician who changed public opinion is Donald Trump. One could argue Trump uncovered or unified grievances that were already there: I challenge them to name a time anything ever changed, without their being an underlying trend. It’s accurate to say Trump changed public opinion.

In the 2016 election, Trump took public opinion in an anti-institutionalist direction, which greatly impacted polls’ accuracy. Quite simply, his supporters stopped answering the polls, producing a great deal of fog of war that interfered with his opponent Hillary Clinton’s ability to assess the state of the race, while Trump relied on narrowcasting and actively changing public opinion to expand his vote share. His “we’re winning” bravado proved greatly motivating to his voters, while Hillary’s emphasis of a vaguely close and highly dangerous election proved less motivating.

Polls have largely recovered from these attacks, and understanding how requires diving into methodology in the next part of this series.

Conclusion

Viewing polling as a cycle consisting of production, reporting, and responding to polls shows how polling is an institution that thrives under high institutional trust and must work to protect itself under low institutional trust. The public plays a role in all parts of this cycle. The faster things are changing, the greater the need for polls, and the greater the challenge to do them well. In the years ahead I think a healthy polling industry is vital to the health of our democracy. Democracy will live or die by public opinion aligning with democratic values, and by changing public opinion to better align with democratic values in an informed way. We should not be fighting this battle in the dark.