Confused and dismayed by the recent articles about non-probability samples?
Trying to understand, “How bad is it? Can I trust data from non-probability samples?”
Then read this:
The Pew Research Center generously shared results from its analysis of nine online panels, and examines response consistency across these non-probability samples. Bottom line: Pew did find some variation by question. For example, “Uses Internet Daily” varies widely across the 9 panels. Marital status, and some (not all) demographics, are far more consistent.
Take-away: REALLY know your online panel providers. And you will be best off standardizing on one or two select partners, to avoid unwanted “noise” in your data.
2 comments
In the end, all polling, be it political, MR, academic, etc. is non-probability. No one is compelled to take a survey, all surveys are opt-in, thus no sample is a truly a random sample, in the strict sense of the word.
Completely agree, not all panel vendors are created equal. Understanding the ins and outs are essential. Some are good for B2B while others you should use only for consumer data. Know your options, thanks for sharing.