Hello research experts!
I'm currently performing research into different types of survey designs, response coding and analyses. I came across a citation I was hoping someone might be able to decode:
"When creating response scales, it's important to take into consideration its orientation. While there are many caveats and exceptions when creating response items, one effect is that respondents tend to favor the left side of a response scale. Take the following two response options:
Strongly-Disagree Disagree Undecided Agree Strongly-Agree
Strongly-Agree Agree Undecided Disagree Strongly-Disagree
If you code the values from 1 to 5 for the first scale and 5 to 1 on the second scale then you’ll have a higher average score on the second response option".
I'm just slightly confused as to how the average score would change in the second response item scale, since it's been inversely coded from the first, so my thinking is that the average shouldn't change, but there may be something crucial I'm missing.
Any help would be greatly appreciated. Thank you!
.
Darko, the average does not change because you reverse the values from 1 to 5, to 5 to 1.
It changes because - as per your citation - respondents tend to favour the left side of a response scale (applies to cultures only who read and write from left to write). In your first option you would expect (slightly) more Strongly-Disagree and Disagree answers, with the second option (slightly) more Strongly-Agree and Agree answers. In other words, you are likely to introduce a systematic error, in addition to the fact that with a 5-item scale (as in any other odd-numbered scale) you have a "tendency to the centre", i.e. undecided. Put it to the "lot of the researcher" .. Of course, if you reverse the values your average won't change anyway, but the bias problem remains.
If you want to make a thorough job out of it, then you use two sets of questionnaires and allocate respondents randomly (followed by a discussion of your findings) - this surely would give your study an extra bonus, practically nobody bothers.
Incidentally, using an uneven-numbered scale may introduce another problem: it's likely that less people answer the question.
Which is why you should measure your instrument's reliability, Cronbach alpha comes to mind, or the split-half method.
Good luck!
Hi there,
To make things consistent, stick to the same answer scale & coding throughout your survey (start with either agree or disagree).
An uneven number of options (i.e. where a central point is present, like in a scale your present - a neutral option) would make people choose that neutral answer more often.
on the other hand, an even number of options (i.e. without a central point) would push them choose either a positive or a negative side.
The choice is yours.
If you are really interested in how the scales work, try it yourself and present different scales in a pilot survey where you interview / talk to your respondents on the spot. That might give you a better sense of theory. Repeat it with at least 10 test respondents.
good luck!
Cheers, Marija
The size of the the correlations with these two versions of an item would be identical, but the signs would be reversed.
Frequently, items with Likert scoring are combined into a scale, and it doesn't matter which way the questions are worded, as long as they are all worded in the same direction (i.e., the correlations need to be positive, not a mix of positive and negative).
Dear Darko,
you can also include several, slightly different questions around the same issue into your questionnaire, and have the same scale running from 1 to 5 in all of them. Just change the perspective in the questions or statements into opposite ones. Let me explain this with an example: You could have these two statements in your survey (a real-World example from our institution)
1. Larger municipalities are more cost-efficient : 1-strongly disagree 5-strongly agree
2. Smaller municipalities listen better to the wishes of the inhabitants 1-strongly disagree 5-strongly agree
We tried to single out the preferences of the inhabitants to the municipal structure, so we had a myriad of questions constructed so that around half of them we pro-small and the other half pro-large municipalities.
Good luck,
Jokke
I support Jouko's suggestions.
Consistent "wording in the same direction" is not necessary, all statistical software packages have the function to invert the coding.
Don't overdo it with inverse coded questions, though. Presently I am using a standardized questionnaire with 28 items, six of which are inversely coded. It is also a check whether respondents answer mindlessly.
The idea of "reversing" items to force respondents to pay more attention is frequently not worth the loss of data that it causes. In particular, if someone does not notice the inversion, answering backwards will lower the overall reliability for the sample, as well as mis-scoring that individual for the Item(s) in question.
There has been a considerable amount of research on this subject, and unless you have a very large, it takes only a few respondents who to make a notable difference..
Thank you all for the very helpful responses! This should definitely steer me in the right direction.
- Badges
- Science method
- Similar topics
- Mathematical Sciences
- Statistics
- Applied Statistics
- Survey Design