Even if a market research project produces a pile of perfect data, we still face the fundamental challenge of analysis — making sure that we’re analyzing the results comprehensively and objectively. In other words, without bias.
Let’s say you’ve done an online survey. You identified your objectives, thought carefully about sampling, and designed a great questionnaire. You monitored data collection and carefully cleaned your dataset. Even after all this painstaking work, risk still exists. You still have to analyze the data, and it’s here that unexpected errors often creep in.
WHAT WE SEE
Human beings have a natural tendency to look for and see things that confirm our hypotheses, or that are consistent with our personal experiences. The result? It’s as though someone has applied a highlighter to the data so that the things we expect or believe jump out at us.
WHAT WE DON’T SEE
We tend to overlook things we don’t expect or don’t believe in, and have an unconscious desire to ignore things that are inconsistent with our experience and expectations. Amid the flood of tables, charts and graphs produced by our survey, there may well be unexpected results that are more significant and useful than anything we anticipated, but because we’re not looking for them, there’s a good chance we won’t see them. If you’re a doubter, take a look at this video.
ATTRIBUTES OF A GREAT RESEARCHER
Some people are truly gifted at market research analysis. Some key signs?
- They’re able to challenge their own assumptions
- They’re willing to play devil’s advocate and challenge the way other people are looking at the data too.
- They recognize unexpected themes that sometimes appear in a data set.
- They resist the temptation to embrace the first story the data reveals; they’ll look for multiple stories so that they can determine which ones have truly compelling value.
- They can deliver bad news, such as “You know, your baby is ugly”; meaning that your favorite project, concept, or idea is getting negative results.
The ability to objectively and thoroughly look at data to see unexpected patterns is the key to being a great market research analyst. Even when a research project is perfectly designed and executed, there’s a real risk that it will fall apart at the analysis stage, wasting time, money and an opportunity to profit from hard work.
Market Research Bias Problems
In any analysis project, researchers need to sanity check for the following challenges:
- Positive Bias: The unconscious tendency to see what we’re searching for or expecting to find.
- Inattentional Blindness: An unconscious tendency to miss what we’re not looking for or don’t expect.
- Happy News Bias: The inability to acknowledge that there is some “bad” news.
If these problems are at all evident, consider the following options:
- Challenge Assumptions: If you see results that closely match your expectations or confirm what you already know a little too neatly, challenge yourself. Embrace the discipline of being objective, or hire people who can be.
- Develop a checklist: Develop a personal checklist to help you maintain objectivity. With experience, you’ll discover where your own biases tend to surface. A checklist of those biases can help you avoid objectivity traps.
- Hire those with the skills you lack: Someone on the analysis team has to be able to see what the others miss, and be willing to speak up about it. Hire people who see things differently, and create an environment where they are willing to be vocal about what they see.
- Diversify Analysts: Different people have different biases and will have different interpretations of a data set. With a team, it’s possible that you’ll cancel out each others’ biases. We each have our weaknesses and blind spots, and a team approach will help to mitigate them.
Market Research Analysis: Did You Catch a Fish or a Boot?
Embarking on a marketing research project is a lot like fishing. You can pick the day, the time, the spot, the lure and the line, but you can’t control what you hook. Before you head back to the dock, take a good look in your hand and see if what you’re holding is a five-pound trout — or an old boot.
[Do you have staff that could use some market research training? Check out our self-paced online classes; most are under an hour, and all can be viewed conveniently from any web browser.]
