As simplistic as it is, market research is often defined as either “qualitative” or “quantitative.” In conversational use, most researchers use and define these categories fairly similarly. But this is changing. While “qualitative” use and meaning is fairly consistent, we are seeing a notable fragmentation in what the phrase “quantitative research” means to market research practitioners and related professionals.
How Do You Define “Quantitative Research”?
I don’t mean a Wikipedia definition; I mean how do you personally define it when asked by colleagues or clients? What do you mean when you use the phrase?
Common real-world answers we typically hear, whether we agree with them or not, include:
- Quantitative research is a set of methods and tools for measuring attitudes, behaviors, emotions, and values. It includes data collection and analysis.
- Quantitative research is survey research.
buy vilitra online buy vilitra no prescription generic
It requires a sufficient sample size to generate results that are representative of an intended population and that would be replicable if tested.
- Quantitative research is the collection and analysis of numeric data in order to test stated hypotheses.
- Quantitative research is any research based on 50 or more participants. Data collection could be from a survey, interviews, or even focus groups[1].
Are these all different? Yes, however still somewhat related. But today, we often hear unexpected variations. Just last week I met a sharp market research professional, who defined quant as big data analysis—he grouped survey research with…wait for it…qual.
Shifting definitions can also be seen in recent job postings for quant researchers. Increasingly common requirements include data mining, text analytics and even HTML.
Quant under Fire
To some extent, fragmented meanings associated with quant research reflect the shift from data collection plus statistical analysis, to an increasing focus on analysis of existing or continuous data streams. But is there more to it?
Some of the fragmentation in what quant research means also seems to be coming from differently educated practitioners—market research newcomers who are from a data analytics or tech bent. We market researchers now commonly work with people who have different skills, training and jargon.
But some of the fragmentation is coming from within the industry.
Two powerful topics were recently written about—tackling some brutal quant research realities. Realities that do have implications for what the term “quantitative research” really means today.
- Articles on whether margin of error should be reported when based on non-probability samples (see posts by Annie Pettit, Chief Research Office at Peanut Labs, and related articles). This raises the issue: can replicable, valid data come from a survey that uses non-random sampling?
- Data quality is a hot issue for quant researchers. A May 2015 article from Quirk’s covers new online sample quality guidelines from ESOMAR and the Global Research Business Network (GRBN). While online sample quality is certainly not a new topic, it is becoming more urgent.
Quant is also picking up some heat from high-visibility polling erros. Most recently, the British Polling Council’s inquiry into the unsuccessful 2015 opinion polls that failed to predict David Camron’s conquest.
Even Nate Silver has said, “The world may have a polling problem,” citing other recent polling failures.
What Does “Quantitative Research” Promise?
Not only is the term “quantitative research” used differently by different people, and under fire, even within the market research profession there are questions as to exactly, “how quantitative is quantitative research?
”
- Just because data is collected using a questionnaire, does not mean the research is quantitative.
- Just because a researcher delivered statistics, doesn’t mean they are reliable.
- Just because a margin of error has been calculated, doesn’t mean it should have been.
When we researchers say “quantitative research”, what do people hear? A promise of representativeness? A measure of reliability? A promise of robust statistical analysis? The use of questionnaires? An assurance of large sample sizes?
The harder question is, what do we want it to mean? As a profession, what do market researchers want “quantitative research” to mean, and what are reasonable assumptions to make when we hear our colleagues use the term?
The Limits of Shorthand, the Promise of Clarity
Today the phrase “quantitative research” means different things to different people. That’s beyond our control. But as market research professionals we should have clarity—what do we mean? The next time a client asks us to define it, what will we say? And can we say it in a way that is meaningful and memorable so that we can help others, and our industry, communicate and manage expectations appropriately? I don’t have the answer; but I do want to make sure we start the discussion.
Kathryn Korostoff is the Founder and Lead Instructor at Research Rockstar. She welcomes comments on this post.
———————————————
Summer Camp for Research Rockstars is now open for registrations. A class bundle normally sold for over $4,000, available for summer only at $950. Want to develop skills with new online methods? Looking to add some analysis muscle to your reports? Take as many of our summer classes as you please. Includes our new bestseller: Infographics for Market Researchers.
Register by June 1st and get the gift of illumination: a Research Rockstar flashlight 😉
5 comments
I just signed up for Camp Rockstar. I’m looking forward to it, especially the new infographics class, and my free flashlight 😉
“How many focus groups do I have to do to get quantitative results?” How about “none” – since focus groups are not based on a random sample of the population of interest.
To your argument about the redefinition of quantitative research, Kathryn, I would counter-argue that it isn’t worth bothering to redefine “quantitative” as both “quantitative” and “qualitative” will soon be outdated. The abundance of data – both numeric and textual – in our world now, with more to come, is a driving force for using less “siloed” definitions. As researchers and analysts our principal function is to provide answers to business problems. We need to be able to do this faster and more effectively, using whatever methodology or combination of methodologies makes the most sense in the situation. I believe hybrid qual/quant approaches will be the next wave.
Kathryn, thanks for this thought-provoking piece. The situation is not all that different from the one posed in philosophy of science. Scientists tend to be muddle-headed about what science is, and clarity was not achieved by historians of science (who say that science is what scientists did) or sociologists of science (who say that science is what science does). I define quant as research based on counting things. It doesn’t depend on the methods employed to count. If I do a wave of in-depth interviews and then code and count themes, that is quant. Of course, I have to include a raft of disclaimers with the results, but they are numerical. But if I take those same interviews and list the most interesting, articulate or useful themes without counting them, then that is qual. Also, you ask if “…replicable, valid data [can] come from a survey that uses non-random sampling?” Absolutely! A complete census is perfectly replicable, and while we can argue over whether it is a probability sample (It is. Prob. of a sample element being chosen = 1), I think we can agree that it is not in any meaningful way random. And why not report the margin of error. It’s perfectly calculable. It is zero! Now this pin-headed way of arguing doesn’t settle the matter because a census hardly represents the kind of research we do every day; however, it does show that some clarity (useful or not) can be achieved by going back to definitions and first principles.
So long as the minimum sample size requirement is met.
Minimum Sample Size = (Z-score)² * StdDev*(1-StdDev) / (margin of error)²
“How many focus groups do I have to do to get quantitative results?”
Enough to encompass everyone (underline everyone) in the target under consideration, i.e. the target population. Could be millions of people.