Since we’ve just completed an election cycle, with its flood of political polling data, let’s take a look at what market researchers can learn from our not-too-distant cousins: political pollsters.
On the surface, political polling and market research do seem closely related. Both make use of survey techniques, data collection and statistical analysis. But there are also significant differences which make political polls quite distinctive:
- Speed: Polls are done quickly (often in a day), while many market research survey projects take weeks or even months from start-to-finish.
- Bias: Polls are often unashamedly “partisan”, and are frequently sponsored by one side, while market research strives (in theory) to be objective.
- Method: Where polls frequently rely heavily on phone-based data collection (though this is changing), commercial market research uses a variety of data collection modes.
Does Polling Have Value Beyond Providing Convenient Sound Bites?
So why talk about polling as a market research learning opportunity? For the same reason scientists study mice or primates as a way of understanding humans —to learn from what they have in common. So let’s look at two important lessons from the 2012 presidential election and see if they apply outside of the polling laboratory.
1. Content Curation Has Value. Content curation is the process of taking content from multiple sources and assembling the elements of interest for a particular purpose. Think of it is as an intellectual blender, mixing various content ingredients to produce an easily digestible, nutrition-packed smoothie. The idea has been a hot topic among marketers for years, and a number of popular bloggers and marketing consultants do it routinely. Now the practice has been brought to the research world, thanks to the highly visible Nate Silver.
If ever there was a Market Research Rockstar, it is indeed Mr. Silver. He’s developed a methodology that shows the importance of aggregation as a core component of data analysis. By taking a poll of polls, aggregating the results, and applying some analysis and judgment to the process, Mr. Silver has done very, very well — his predictions for the state-by-state results were perfect. Yet one could argue that he creates no original research; which is hard to swallow for many of us who enjoy and take great pride in primary research.
Silver’s success raises an interesting question; are there opportunities in market research for “poll of polling” methodologies? Granted that political polls are somewhat unique by virtue of the vast number of polls being taken simultaneously on the same topics, but could it work?
Instead of one large project of 5,000 respondents, what if we did 10 smaller surveys from 10 different sources? Maybe even have them managed by 10 different teams? How different would the results be? Would our “survey of surveys” aggregated results be different than those from a single study? Would they turn out to be more insightful? And if predicting an outcome, more accurate?
2. Prediction Markets Rule. I’m a big fan of prediction markets. It’s a methodology with enormous potential for the market research industry and it’s a topic I teach and demonstrate in our Online Research Methods class. Skeptical? Consider this: both the Iowa Electronics Markets and the Intrade Market correctly predicted Obama’s win months before the actual election. Prediction markets are particularly interesting in this case because not only were they accurate, they had less variation than one might expect. The “wisdom of the crowd” that’s tapped for prediction markets concluded that Obama was going to win early on; it wasn’t like they all converged on the right answer three days before the election. If the “crowd” can predict presidential election results, why not which brand will ship the most TVs in January? Or which market segment will purchase the most tablets in 2013? Or which brand of mobile phone service will have the highest renewal rates in 2014? Check out Huunu from ConsensusPoint for an example of bringing prediction markets to market research.
What Can Market Research Learn From Political Polling?
In market research, we rarely have the benefit of getting a known, quantifiable outcome, and here, political polls have an advantage. You know who won, you know by how much. That’s less common in market research. We do a study that suggests that a product configured with certain attributes will be preferred, but generally can’t test that result perfectly. Similarly, a project might identify which marketing messages will resonate with target customer groups, but we’ll never know precisely how “right” we were. But, there are exceptions, such as the hypothetical predictions mentioned above. So even with the differences, or in spite of them, political polling may indeed offer a sort of laboratory.
Many genetic studies use mice because of their genetic similarity to humans (yes, really) and also because they are easy to breed (volume) and control (replicate). By looking at political polls as our figurative mice, perhaps we can develop prescriptions that will make market research healthier.
[Want access to more market research articles and training materials? Sign up for the Research Rockstar newsletter: SIGNUP]
2 comments
Marketing researchers do practice content curation in many guises, particularly on the client side, where non-researcher stakeholders often want big-picture thinking that connects the dots between studies and combines different types of data. Cambiar has been using the metaphor of “The River” at marketing research conferences for several years now, and the need to curate data from many streams is implicit in this concept.
Many client-side teams perform annual or semi-annual syntheses of data, often to prepare for major sales or strategy meetings. Quantitative, qualitative, secondary, and other sources of data are combined. Consistency across studies suggests convergent validity, which is reassuring. A few of my clients ask for help in integrating the many studies we do for them.
On a smaller scale, we often start a project by reviewing what our client already knows to pinpoint the knowledge gaps that require new research, or data/hypotheses that need to be confirmed. I’m not shy about citing somebody else’s study if this provides useful contrast, validation, or elaboration of the current study.
Doing a “poll of polls” is akin to meta-analysis, which is sometimes done by large corporate MR departments, where there are sufficient studies and technical expertise (particularly staff statisticians). New possibilities now exist for big-data analysis, leveraging technology resources in the cloud.
Some client teams curate libraries of research that can be searched, thus avoiding inefficient duplication of research. (One person’s inefficiency is another person’s scientific replication and search for convergent validity.) A library can provide grist for the meta-analysis and synthesis mill. For example, my team has managed a global library for consumer product manufacturer for many years. A major pharmaceutical company has a policy requiring a search in their library before approving new research.
On a smaller scale, normative databases for a specific type of quantitative research is a narrow form of curation.
Content curation is a very useful idea that is already proving its worth for marketing researchers. In a world where non-researcher client audiences want big-picture thinking about the business and MR departments need to do more with less, curation should become a routine way of life for us.
Thank you for the very thoughtful reply! I will Tweet this so others can benefit.