News

By Hena Naghmi, Senior Consultant

We’re two days on the other side of a presidential election in America and we still don’t have a clear winner. While many pundits and analysts tried to prepare us for this very situation, warning of the potential of a longer vote counting process due to unprecedented levels of mail-in voting during a pandemic, polls nationwide told a different story. We’re all hearing the ghost stories of the ABC / Washington Post poll that had former Vice President Joe Biden at a 17-point lead in Wisconsin. Or FiveThirtyEight’s bold analysis that President Trump had a mere 10% chance of winning the election. Well, as we know now, Joe Biden barely eked out a win in Wisconsin and Donald Trump still has a chance of winning this election.

If you’re like me (or my colleague Patrick), you went into election night skeptical about these polls. Not only because being skeptical makes you seem cool and informed, but because you didn’t want to experience the whiplash of 2016. But early into Tuesday night, that whiplash is exactly what all of us were experiencing. With races in key swing states that did not at all reflect what pre-election night polling told us, we were all left pondering the same question: how did pollsters get it so wrong again?

This year the pollsters had promised us they’d fixed their methods and models and that a more nuanced, state-based approach was being employed. Yet they got it wrong once again. So what did they miss? They fundamentally underestimated the extent of nonresponse bias in their methods. That is, the extent to which the people who opted in and decided to respond to the survey differed in ways meaningful to the survey subject from those who opted out and decided not to respond. So for example, if your survey subject was about dogs, you would likely get more people who like and / or have dogs to respond to your survey which would artificially skew the data. In the political world, people who tend to respond to surveyors are likelier to trust the survey institutions and believe in the value of political polling. Usually this type of systemic bias can be overcome by increasing sample size. But clearly that did not work in 2016 nor in 2020. The reason that merely increasing sample size did not overcome nonresponse bias is because the subject being surveyed was uniquely sensitive. 

Politics As Identity

The past 4 years have cemented the idea that your political party preference in our two-party system, especially who you voted for in 2016 or 2020, is the foundation of who you are as a person. So much so that in the days leading up to election night, NYTimes posted a lighthearted quiz challenging readers to distinguish between refrigerators belonging to Biden and Trump-voters. People were genuinely surprised at how difficult it was to easily discriminate these fridges (I personally scored just around 50% on the quiz which is as well as I’d expect to do if I guessed randomly). They were shocked that presidential political preferences did not easily extend to food selection or refrigerator cleanliness levels. And that shock only makes sense in the paradigm of “politics as an identity” that has become the norm.

The way political preference has been seamlessly weaved into identity and sense of self has made it particularly difficult to employ empathy when communicating with someone whose politics differ from yours. At RedSky, we talk about well-structured survey instruments as conversations. If you have a well-written survey or discussion guide, it should feel like a conversation between the surveyor and the survey-ee. And herein lies the problem. If the surveyor is seen as an agent of a particular political party or an institution that biasedly supports a given party, members of the opposite party won’t be particularly open to that conversation. And as difficult as it is to employ empathy in conversations with someone in your life whom you know has an opposing political view, it’s even more difficult to do so with complete strangers with whom you lack a personal relationship and whose identities can therefore be reduced to their political preference.

Employing Empathy

So as a surveyor, how do you overcome this issue? Is it even possible, or should we just throw in the towel and say “we’re done trying to accurately assess the hearts and minds of the American people when it comes to sensitive issues?” I personally think that seeking to understand is always a worthy endeavor so to the latter I say no thanks. I think the answer lies in empathy. Empathy is a cornerstone of our surveying principles at RedSky, and the heart of our HumanSight philosophy and approach. When we think about employing empathy in our survey instruments, we take into account how we will be perceived as a surveyor. Fortunately we don’t have any of the baggage that comes with being viewed as a political pollster. But even so, we know that groups that are giving the surveys don’t often get the benefit of the doubt. So we lead with transparency. “Here is why we are asking you these questions. Here is who will see your answers. Here is our role and relation to the subject matter.” Leading with transparency is the critical foundation.

Next, you must ensure that the person taking your survey understands that this particular conversation is an instrument they can use to make their opinion heard on something that matters to them. In the context of market research, remind them that they were specifically selected for your survey. That their unique opinion is invaluable to an organization and could help shape the business strategy of a company that makes things they care about and use. Frame their participation in this study as an input to making that product or service better. 

Similarly when it comes to political surveys, taking a couple of minutes or a bit of whitespace up top to remind your participants that responding to a political survey is a way to let their voice be heard and potentially shape everything from campaign strategy to voter turnout might make them feel more emboldened, empowered, and willing to participate. Reframing the survey instrument as a tool at the respondent’s disposal to help them shape the world is a form of empathy that all researchers can employ.

As we all reel from the whiplash of polls gone wrong yet again, we should remember that there is hope in empathy. That empathy is a critical component of any successful interaction between two human beings. And at the end of the day, a good poll should be nothing less than that – a conversation that leaves each party feeling heard.