We all know what the most common survey design mistakes are: having surveys that are too long, too onerous, or that have questions that are leading or biased in some way. But what about the next most common survey design errors?
1. Too many open-ended questions. Very often, when we’re writing a questionnaire, we realize that there are opportunities to discover new things or we are unsure of which answer options to offer. Our solution? Open-ended questions. We might ask people about unaided brand awareness; that is, “When you think of Product Category X, what brands come to mind?” Or you might ask, “What else can our company do to improve your satisfaction with our services or products?” Or you might have a question followed by a list of possible answers, including an “Other. Please specify: ” option. The first two examples are open-ended questions, but even the third one is expecting a lot. Having two or three questions that require real writing is fine. But if you ask too many, it becomes a turn-off. It is simply too onerous and few respondents will type that much.
The result? You end up with a lot of missing data. So choose wisely, and use open-ended questions judiciously.
2. Excessive jargon. If you’re doing a survey project, chances are that you have a great deal of expertise in a particular product category, industry, or topic area. And by virtue of the fact that you’re an expert, you have developed a specific language for talking about relevant issues.
It’s very easy for those of us who develop areas of expertise to forget that other people simply don’t use the same language to discuss the same topics. We have to be vigilant when we’re creating surveys to use friendly language. Go for the lowest common denominator in terms of who’s going to be taking your survey—and use language that they are likely to use. Excessive jargon turns people off and leads to dropouts, or worse. If they don’t really know what a term means they might guess, and you might be getting inaccurate data in return.
3. Forgetting your manners. It sounds trite but it is really true. We need to be respectful of the people who are taking our surveys. An occasional “please” and “thank you” goes a long way. In the survey opening, use polite text to set the context and invite them to the survey. Remember, they’re doing us a favor. At the end of the survey, there should be a clear and distinct thank you message, especially if this is a survey going to your own customers. I’m stunned at how abrupt many surveys end.
If I’m a customer and I’ve just given you 5, 10, or maybe even 15 minutes of my time to answer your survey, and it simply ends at the last screen, that’s not really very nice.
Here’s some possible text: “Thank you. Your opinions are very important and will help us to improve our products and services.” Or, “Thank you. Your input has been extremely valuable. Stay tuned to our company newsletter to hear how we’ll be applying these important research results.” Let them know that it wasn’t just an academic exercise; that you plan to actually use the research.
While it is great to see that there are so many free and low‑cost survey tools available today, such as Ask Your Target Market, SurveyGizmo, QuestionPro, SurveyMonkey, and Zoomerang, there are lots of mistakes that people can easily make when writing surveys. Finding a great tool may be easy these days, but writing a great survey is not.
Planning to write a questionnaire? Let Research Rockstar show you how to manage the process and avoid common mistakes. Click here for details.
5 comments
Good points on survey design. It’s also a good idea to start with a top down rather than bottom up approach by deciding what information you need or would like to know before composing the questionnaire & collecting the data.
There’s also a great alternative survey tool to the ones listed. It’s called Rational Survey and has plenty of advanced features & a great free plan for students or small businesses with some generous usage (1000 respondents / 100 questions / 10 surveys).
http://www.rationalsurvey.com
I agree with all your points. Questionnaire design is an extremely important part of the process – if they are not designed correctly, the data or insights you derive become far less meaningful.
I would also incorporate survey planning and impartial analysis to the list of things you need to do properly to ensure you get real and accurate information. It may not always be what you were expecting, but getting valid data is the first step in understanding what your respondents are thinking!
These are very good points. If I may add another design aspect to consider, it is the survey length. With the growing usage of smart phones, if your surveys have too many questions then it may turn off your mobile respondents.
You can use a cross-platform tool like http://MoboSurvey.com to create a survey, and it automatically optimizes for smartphone, tablet and desktop. Its native look and feel give your users an easier time to fill out the survey. So it’s definitly important to have only an appropriate number of questions for each survey.
I agree that it’s a great idea to get started with a top-down approach instead of bottom-up approach. Also, my personal experience working with http://www.surveyrobo.com/ says that one survey question should lead you to the other. So there has to be a logical sequence. Thumbs-up for your efforts here. Thanks.
Thanks for your comment and nice to meet you!