We have a questionnaire including several constructs and each construct contains four questions. I have a discussion with a colleague of mine who believes that construct names should be visible on the questionnaire (as titles on top of the respective questions), while I think such visibility might increase response bias.
For instance, if a construct name is 'organizational culture', and one believes that organizational culture around the understudy topic is great, s/he may answer 5/5 to all the questions of that construct without paying much attention to the questions.
My colleague believes it is standard and common to show construct names. What do you think? I appreciate if you could share any references that you know of to support your thoughts.
Thanks,
MJ
There seems to be a similar question here: http://stats.stackexchange.com/questions/24668/should-i-randomize-survey-questions-if-a-respondent-will-fill-in-identically-str
I also found this paragraph supporting my concern:
"The grouping together of similar items leads to a response bias that results in the subjects' giving similar ratings to contiguous items within the same modality. On the other hand, the destruction of the traditional factor pattern contingent upon randomization of items does give a solution explainable in terms of known sensory function. Furthermore, this pattern is probably much less contaminated with response bias effects and may thus be a more accurate reflection of cognitive organization."
Ref: http://psycnet.apa.org/journals/cbs/10/1/68.pdf > an interesting article!
You can mix all questions, they do not need to be under a special subtitle
There seems to be a similar question here: http://stats.stackexchange.com/questions/24668/should-i-randomize-survey-questions-if-a-respondent-will-fill-in-identically-str
I also found this paragraph supporting my concern:
"The grouping together of similar items leads to a response bias that results in the subjects' giving similar ratings to contiguous items within the same modality. On the other hand, the destruction of the traditional factor pattern contingent upon randomization of items does give a solution explainable in terms of known sensory function. Furthermore, this pattern is probably much less contaminated with response bias effects and may thus be a more accurate reflection of cognitive organization."
Ref: http://psycnet.apa.org/journals/cbs/10/1/68.pdf > an interesting article!
There is a difference between telling the respondent the construct name and grouping questions under a 'neutral' heading which does not make the construct explicit. If you do the former then you are creating bias by giving clues as to what sort of responses are of most interest to the researcher. If you do the latter then you may be creating some 'response set' bias but that is offset by making the survey completion a more meaningful and 'holistic' experience (because the way that the questions are arranged 'makes sense' rather than being 'random').
I think that is not necessary to put the name of the construct at the beginning, because certainly it would generate a bias only for social desirability. If you need some organizational structure, you can organize your questionnaire replaced the name of the constructs for their initials, in this way you can understand the structure, and the participants understand that there are a new section or group of questions.
About mix all question, i think that is not a good idea if you are using some standardized instruments, there are information about this and the possible advantages and disadvantages of mixed it, but in my personal experience y prefer to leave the questionnaire in the way that is propose for the author of this.
I find the idea that knowing what your questions are about could cause bias to be rather strange. For example, if someone thinks that their organizational culture is good, then you want your responses to reflect exactly that.
That said, my impression of standard practice is that it omits the titles. Even so, this strikes me as mostly an issue when your titles are ambiguous and thus open to different interpretations. In other words, you don't want some respondents to interpret your questions as about one thing while other respondents think they are about something else.
Overall, confusion in responding is a far worse problem than response bias. The reason why people group similar questions together is precisely so that respondents can understand the topic for the full set of questions. In particular, answering even a single question "backwards" due to misunderstanding will damage your reliability -- which is why so called "reversed" items have gone out of favor.
Mr Jalali, I think that you make your instrument in both ways: putting construct name or not. In last case, you can mix affirmatives or put in a sequence. When you make an factor analysis, you separated the items.
It very much depends how you will use the questionnaire. If there will be an interviewer then it is good to leave the constructs. That way it will be easier for the interviewer to know when they jump through the sections. But if the person has to fill the questionnaire hminself/herself, it is probably better not to leave the constructs. One thing is the bias and another thing - people might just choose to leave some sections out if they do not like the title of the construct. But, if it is an Internet survey where they do not have the possibility to jump forward, then you can write an introduction, like 'now we are going to ask some questions about...., please....' and so on. Whichever way you decide to go, you have to admit the limitations it could bring. Like if you leave the constructs, you might wirte something about it when you discuss your method and justify your choice.
Since measuring "latent variable" (also called construct with multiple indicators) is the issue here, it should be "latent" in the eyes of the respondent, otherwise response bias would creep in.
It depends on whether you want to reveal or not. Also you may mix up the questions without disturbing the theme so as to make it comfortable to the respondents.
not necessary a general survey, but for some specialised survey you may need to include
The names of latent constructs should not be an issue in research. You can reveal the name of a construct and also the name of its sub-construct on the questionnaire.
- Badges
- Science topic
- Similar topics
- Psychology