To my knowledge, most of the research that was done using the Theory of Planned Behavior (TPB), either utilized something similar to what Ajzen recommends (i.e. providing statements in a fixed order, using semantic differential scales, etc.), either used something completely different (e.g. used single statements to measure constructs, or using scales of several items for some constructs, while measuring other constructs using just one statement, sometimes these statements are rated on classical 7- or 5-point Likert scale from Disagree to Agree).
I was wondering, if there are any articles that explore possible response tendencies and question order effects, that may occur when a TPB questionnaire is constructed in different ways.
Also, is there a best way to construct a TPB questionnaire. I understand, that one should strongly consider following Ajzen's guidelines, but are there any criticism directed at the way these questionnaires are usually constructed?
I have a some thoughts. First, if you're worried about order effects, then it's easy using online surveys to randomly order the items and sections and I strongly recommend that you do so simply because then the problem goes away.
I don't know much about Ajzen's work with TPB, but I took a class from Fishbein on the Theory of Reasoned Action (TORA), which is a kind of a predecessor theory (with Ajzen). The TORA was a kind of break-through because at the time of its inception, attitude theories did not predict behavior. So it was a huge coup for Fishbein and Ajzen to use attitudes to predict behaviors. But a consequence, at least for Fishbein, was he became insanely rigid about his model -- to the the extent that one of our class topics (for a 2 hour, graduate-level class) was discussing whether to phrase the BI as "I intend" or "I will".
So, my insight is that Fishbein, at least, had hugely over-sized opinions about minute details. I don't give a fig what Fishbein or Ajzen thinks is the "right" way to do things that are outside the scope of their area of expertise (like the order of the questions, or the specific verb used in the BI). I would use your best judgment and the expertise of people who have studied the specific issue (e.g., context effects in items).
So, I haven't looked at the context effects literature lately but they are definitely possible. The psychometric (or a psychometric) analysis you can do to find such effects (in a survey with fixed ordering) is compute Yen's Q3 statistic to detect local item dependence (LID; See Mislevy et al, 2012). Again, I'm not sure what value there is in finding such effects, I think the issue is removing them and randomization provides an ideal method.
I don't think you will find huge differences from slightly different ways of getting essentially the same information (like semantic differential vs. Likert; 5- or 7-point responses).
Mislevy, J. L., Rupp, A. A. and Harring, J. R. (2012), Detecting Local Item Dependence in Polytomous Adaptive Data. Journal of Educational Measurement, 49: 127–147.
Dear Alan,
Thank you for your answer.
As a matter of fact, I've done some research on order effects, when measuring social norms, and these order effects were quite pronounced.
However, my main concern is whether it is better to formulate several separate statements for measuring, for example, attitudes (as opposed to giving a semantic differential scale to rate one statement).
I fear, that the TPB and its predictive power may largely depend on the inter-correlations between constructs, and these correlations usually are so high because of question order effects and response tendencies. In other words, I'm worried that if one formulates statements, that could be rated in a Likert scale from "Strongly agree" to "Strongly disagree", and these statements would be presented in random order, the fit of the model would substantially decrease.