Article Synopsis: Should I label all scale points or just the end points for attitudinal questions?
E-Journal Survey Practice
By Aaron Maitland
Written by Research Rockstar intern Sarah Stites
Experienced market researchers know that survey design is both an art and science, involving thoughtful consideration of question wording and order, length and scale selection. When it comes to the latter, one of the most difficult questions for the survey researcher is determining how to label scales on attitudinal questions. Should only the scale anchors be labeled, or should each point on the scale have a label too?
According to Aaron Maitland of the National Center for Health Statistics, the point of labeling is to add greater clarity to questionnaires. Thus, some might argue that labeling should be as complete as possible. Yet, although researchers agree that it is more natural for respondents to express their opinions verbally, verbal labels can also be more ambiguous than numerical labels. Plus, when it comes to cross-cultural research, words are more easily misconstrued than numbers. In order to combat this, Maitland recommends that researchers conduct cognitive interviews with survey respondents in order to understand how the latter are assigning meaning to scale labels.
Furthermore, the labeling issue depends largely on model and length of a given questionnaire. If the survey is being conducted by phone, full labeling becomes tiresome. In the same way, longer scales become more unwieldy to fully label, as finding enough adjectives to define each point becomes difficult. Yet, if the survey is visual and scales are relatively short, the precision of full labeling can make sense.
Given that ultimately data quality is the most important issue, the research must decide: does partial scale labeling produce less reliable data than full labeling? According to several studies, this appears to be the case. Maitland mentions a study in which the reliability of 7-point scales with only anchor labels was compared to that of scales with each point labeled. The fully labeled scales had a reliability of .719 compared to .503 for the partially labeled ones. Fully labeling may also increase question comprehension among less educated respondent groups, producing better quality data.
Here at Research Rockstar, our lead instructor Kathryn Korostoff notes, “Obviously there are trade-offs, and further research should test how different labeling strategies’ effectiveness varies by age, education level, and duration. Some companies may find partial labeling is better for their target populations—it’s worth some testing to find out.”
________________________________
Think you could use some more training on questionnaire design? Check out our related course offerings: Questionnaire Design Success: A 10-Point Checklist and Questionnaire Design 201: Scale Selection.
