News

By Anthony Bennett, Managing Partner

There is a parable among market researchers about a woman who drowned in a river that was an average of two feet deep. Indeed, one of the most dangerous metrics used in research reports is the average (research itself isn’t terribly fraught with danger). But we love averages, don’t we? We see them all the time in reports, quote them in presentations, and spot them in the news. What’s the harm in using an average?

Averages are useful, but they have an insidious side, too. They hide data points that provide insight, and they present an attractive veneer of simplicity. Isn’t it tempting to talk about the “Average American” or the “Average New Yorker”? I have no idea who these people may be, but if you see them, please let me know.[1]

Here’s an example of important data hidden by averages — Say we have three respondents who answer a survey about how much they intend to buy a particular product, on a 5-point scale, where 1 = “definitely will buy,” 3 = neutral, and 5 = “definitely will not buy.” Our respondents answer as follows:

• John = 5
• Ringo = 4
• George = 1

The researcher reports that “on average” the intention to purchase is 3.3/5. The product manager reports that the target market is neutral. But are they really? Two-thirds of our respondents won’t actually buy the product.  

Statistics has solved this problem through standard deviation (which measures the range of answers around the mean). But when was the last time you saw standard deviation in a market research presentation? Or heard it mentioned on the news as a caveat to what the “average American” does?

 We see this problem in segmentation studies. Thinking about the example above, we could put John and Ringo in one segment because they exhibit certain attitudes and behaviors. George might be in a different segment with others who think like him. Very often, our clients ask us to combine segments to create larger groups that may be easier to find, and therefore, appear more lucrative. 

By putting John, Ringo and George in one segment, we make the segment bigger, but water down that segment and make it far less insightful. At the end of the day, only George will buy the product — any marketing or media budget towards John and Ringo will have no ROI.

How do you avoid the danger of averages?

1. Ask the researchers to break down the numbers. Averages are great — but how many people rate the product in the “top box” gives a better understanding of purchase likelihood. (In the example above, it would be 33%, or one of every 3 respondents.) 
2. When developing a segmentation, always get to the greatest number of segments possible before consolidating them. If you do this, you will at least understand what knowledge you are giving up by combining segments. 
3. For critical data points, always consider the confidence interval and the standard deviation. If a researcher tells you that 60% of people loved your advertisement, know that the “true” number could be anywhere from 56% – 64% (if your sample is around 400 people), but,  a large number of people might have hated the ad. The average is hiding the truth.

We’ll never stop using averages. They are simple and easy to digest. But never forget: They usually are covering up something. Good market researchers know what’s hiding underneath the average: the opinions of  different human beings, whose opinions each matter.

[1] Actually, the Washington Post did an analysis and found that the “average” American is a white woman, aged 52 years old with a bachelor’s degree who works in education and health services and earns $889.62 per week. She lives in a city and is politically independent and tends to vote Democrat. She has a 26.1-minute commute and is married, living with her spouse (probably her husband) and is an Evangelical Baptist.