Imagine that you’ve hired a market research consultant to do a big survey research project for your company, Acme Tea & Coffee. It’s a really important project, and you spent a lot of money and time to make sure it was done correctly.
The research consultant delivers the results. In their presentation to you and your colleagues at Acme, they report the data in aggregate (total). Let’s say you did a survey of 1,000 people, and the consultant reports the following perceptions:
- 80% think your brand sells high-quality products.
- 60% think your brand has unique products.
- 50% think your brand is socially responsible.
These aggregate results are interesting, but you know that in your survey sample there is a mix of males and females. So, you ask the presenter, “How did these results vary by gender?”
Now imagine in our hypothetical scenario, the market research consultant says, “Well, we’re not presenting the data that way because it doesn’t matter. You need to understand how your whole market perceives your brand.”
And there it is: the big red flag.
Hopefully this scenario hasn’t happened to you, but we recently read a case study where it did happen. And it gives us a great example of a market research red flag. If the details of this case study are accurate, the presenter was simply not qualified. And frankly, it sounds like they were trying to pull a “CYA” move because they didn’t have the data handy.
Of course it makes sense to look at variation by gender! In most consumer studies it is important to look at variations by gender, household income, age or other demographic items. Are there ever cases where it is not necessary? Yes. In the very odd situation of a survey where there is no variability in the data: all of the respondents have nearly identical answers. In 30+ years of doing survey research, we have never seen this. It’s theoretically possible, but we have never had it happen.
In a recent Conversation for Research Rockstars, we share an example from an unfortunate real-world case study and suggest how the client’s reasonable requests could have been addressed by doing subgroup analysis and looking at the variability of the data. Two very simple tasks that anyone who presents themselves as market research professionals should be able to do easily and quickly.
Check out the short video here, or for an audio-only option, access from Apple podcasts.
Surveys can be used to learn a lot about your target market’s attitudes, behaviors and perceptions. But only if you analyze the collected market research data correctly. In this real-world example, we see an-all-too common error. Don’t let this happen to you!
