Today I read a blog that stated, “… the utility of market research is often minimal. Many times the data is worthless even before the survey hits the field due to quickly changing business conditions, and consumers are over surveyed and fatigued by the constant bombardment of surveys online or elsewhere.” The blog is from The Armory, and is authored by Brendan Miller. I like Brendan’s posts—he has strong opinions and clearly enjoys innovative thinking. But given how many blogs and articles I have seen lately that express a similar point of view, I just have to respond.
Look, traditional market research is flawed. We all know that. It has its issues—sample quality being a biggie these days. And Brendan sums up another one nicely, “Traditional research only captures a moment in time.
” True, true, and market researchers are well aware of this (and advise clients accordingly).
But let’s not pretend social media is the elixir…the magic potion to cure all market research ills. Yes, social media as a research tool has real benefits and the innovation is exciting for suppliers and clients alike (tip of the hat on a nice piece to Fresh Networks). I am particularly interested in techniques for monitoring online conversations (nice intro by Beth Harte here, plus do include Crimson Hexagon). But if we set unrealistic expectations about how fantastic social media is as a market research tool, we will ultimately disappoint clients, or worse (deliver misleading or egregiously flawed research).
Yes, it sounds great to make sweeping statements about social media-based research “…like creating online customer forums can help marketers take an active and continuous listening approach. Their insights will be timelier and therefore more relevant.” Timely? Perhaps if people happen to be talking about something you care about when you care about it. But alas, these methods also have inherent limitations and biases.
Heresy, you say?! Yes, social media-based market research has real limitations. Two of the major issues:
1. The Popcorn effect (well, that’s what I call it anyway). When someone is particularly frustrated or particularly thrilled they “pop” onto a blog or user forum or review site and share a comment online. Many online forums suffer from these extremes, so we have to be careful. (In contrast, surveys capture a fuller spectrum of response including neutrals—which are a legitimate response and critical context in many cases).
2. Online personas. How people talk, behave and portray themselves online is very different than how they do these things in-person. Ask anyone who has been on an online dating site and then met the individual in-person; the gap between online and in-person can be shocking. And usually not in a good way. As just one example, in some markets, monitoring online communities would suggest an extremely rational set of buying behaviors backed up by shared reviews and deep, objective product evaluations. But in (gasp!) a focus group, a little discussion leads to people confessing to each other that the tie-breaker between brand A and brand B was based on an entirely irrational input (“I wanted my new HDTV to be sleeker than my brother’s”, the knowing nods of the other group members allowing the moderator to use the group dynamic to probe further and peel the onion on customer behaviors).
Bottom-line
The market researcher who clings to conventional surveys and focus groups like a life raft on a turbulent sea is going to drown. Those who judiciously add various social media and ethnographic-based methods along with some of the other fabulous new qualitative research tools out there will be able to navigate through the storm—and best help clients choose the methods (or mix) for their unique needs. But let’s not pretend that social media-based research is a magic cure-all; too many snake oil salesmen will only ultimately turn off clients and lead to a backlash. And that’s not going to do anyone and good.
8 comments
Kathryn,
I couldn’t agree more. There is no one silver-bullet technique. Each has its own weaknesses and strengths. Researchers who are still clinging to there old ways and not evolving will find their research less and less relevant.
Thanks Brendan–and nice to meet you!
I’ve never understood the personae criticism: surely part of the value of SM research is that precisely that it allows direct access to people’s aspirations and performed selves – selves likely to be more and more important as the gap between private and public life continues to narrow.
But I agree that SM is far from a cure-all. If people are trying to get it to replicate survey-based research findings on the cheap they’re going to wind up in all sorts of difficulty. It’s a new tool, and it’ll be a few years before we’ve really got a handle on the kind of insights it can deliver and the best ways to validate them.
Great to hear from you, Tom. As you obviously know, in any form of research, we struggle with the gap between how people perceive themselves and how they actually behave. And I agree with you–sometimes capturing their aspirations (even if they do not reflect current behavior) is very important. My concern is that too many folks are suggesting that SM research more accurately captures reality than surveys and focus groups. Don’t get me wrong–I am painfully aware of the limitations of traditional research (I’ve done over 600 quant studies and personally moderated hundreds of focus groups)! But I am concerned that non-researchers now using SM to become “market researchers” and even some old timer MRers are over-promoting SM-based methods because they are so sexy right now.
I like that post. And I like the comments.
Only wanted a add my hypothesis concerning “the personae criticis”:
The more www and with this SM ist part of the audience’s life and kind of their media education, the smaller the gap between “online” and “in-person”. So it depends on (what we all know is) belonging to a cohort.
That is in parts what research on gamers tells me. Young gamers’ online-gaming aspirations replaces “old fashion” offline gamers’ (like belonging, connecting). But as they got used to online-gaming, they fulfill their needs online. Online gaming itself is also strongly related to age / cohort.
Great post! I’d add the concern that social media comments areen’t representative. Even when we set up an online community for feedback for a client, we integrate the survey system with their entire customer list, not just the list of community members–they need to reach beyond just the participants in the community, who may or may not be representative of other customers for different topics.
I’d be interested in your thoughts on using SM to innovate with identified promoters, passives and detractors. Using more traditional research to identify drivers of loyalty/satisfaction, and coralling each of these groups to generate ideas for change. Bad research, or appropriate use of SM in concert with other research?
Frankly, I am all in favor of using multiple methods as you suggest. In fact, a real problem I see in most market research projects is that they are based on a single method–no triangulation with other data, no sanity checking with other sources. I have been guilty of this as well–there have been times when I have fallen so madly in love with my data from a large quant study, that I didn’t even think to gather complementary data from other sources.