I am investigating a new topic with an established questionnaire. However some items are not suitable so the scales are not the best predictors in my hierarchical regression (n.s.). However, some other items of another questionnaire are far better and belong to the same scale (rated by five idependet experts).
This leads to the idea to develope a new questionnaire that is a mixture of both.
Unfortunately both questionnaires have different Likert scales (5 vs. 7). The endings are the same but not the number of graduation. Is it still possible to put them in the same scale if they are z-standardized.
Thank you for your help.
Hi Julian,
I recommend converting all items to the same response scale. First, there is the possibility that score variances may be (slightly) greater for the 7-point items than the 5-point items. However, this notion is disputed by some researchers since it is not always the case. Second, this will make the questionnaire less confusing for participants. In practice, self-report measures should be as clear and intuitive as possible.
The question now becomes: to use a 5-point or 7-point scale? Here is another thread addressing this very issue: https://www.researchgate.net/post/Should_we_use_a_5_or_7_point_Likert_scale_Whats_better_and_why
Personally, I doubt it will make much difference using either a 5-point or 7-point scale for all items.
Best,
Blaine
Hello Blaine,
thank you for your answer - that helps alot! Also the 5 or 7 point discussion is very interesting.
The variance is the problem I am facing. My data are already collected and both questionnaires have been used in the style they were created. That is the reason for the two different scales.
While the 5-point scale questionnaire is the "gold standard" and center of my hypothesis, the 7-point scale questionnaire was just for exploratory purposes to examine what it explains additional.
After all it turns out that the second questionnaire is not a addition but more replacement for some subscales of the first. This pushes me to the idea for a new or improved questionnaire out of both.
However, I am confused if the R² of the 7 scale in my hierarchical regression is somehow over-estimated due to the fact that it has seven possible values and therefor is so effective as additional block.
At first yes ... but this will mean a serious correction problem and a possible Bias DUE TO ADDED DIFFICULTIES TO SUBJECTS WHO HAVE TO COMPLETE THE QUESTIONNAIRE FOR MORE THAN POSSIBLE CONFUSIONS! ... All this without ENTERING INTO THE STATISTICAL TREATMENT IN THE THAT A SCORE IN THE 5 ANSWER OPTIONS DOES NOT EQUAL TO THE SAME WHEN THEY ARE 7 How would you solve it? My humble advice is to homogenize the answer options: 5 ARE MORE THAN ENOUGH!
Hi Julian,
In this case, I see two tenable choices moving forward. The first option is to drop the exploratory 7-point items and conduct your analysis with just the 5-point measure that was part of your original hypothesis.
The second option is to conduct two analyses: One with the original 5-point measure that was included in your original hypothesis, and a second exploratory analysis with the second (7-point) measure. It was always drilled into my head as a grad student to never mix confirmatory and exploratory measures/analyses. Keep these separate and clearly identify each as such. If you opt for this option you can run separate models (one using data from each questionnaire) and make a direct comparison.
In answer to your final question, it is not so much scale width that influences the R2 of a model, moreso the number of variables or number of items (depending on which type of analysis you're running).
-Blaine
Hello Julian,
Yes, you may combine items that have differing numbers of response options into the same scale. The chief concern is, do they measure the same thing (e.g., are they unidimensional), or would you be better served by having subscales? However, I wasn't fully clear on whether you were using individual item scores as variables, or (as is typically preferred with Likert-type measures) using summated or averaged scores across a set of related items as variables. Factoring might be useful here, if composite/combined scores were to be used.
As for an instrument which had some 5-option response items and some 7-option response items, the arithmetic doesn't go awry because of the difference. It's not essential to standardize items before using (any correlational based technique, such as regression or factor analysis invokes standardization anyway as regards the "beta" coefficients or item-factor correlations). It would be a good idea to separate the two sets in the measure/survey/instrument so that respondents complete all of one type first, then are presented with the other type.
As well, it probably would make little practical difference if you amended all of the items to 5-option or to 7-option type. Yes, there's literature about whether one is preferable to another with respect to issues such as: (a) estimates of score reliability (across a set of items, not for individual items); (b) whether more options makes the resultant scores behave more like a continuous variable; and (c) whether avoidance of extreme score choices might be more prevalent with one than the other. I don't see much practical impact to routinely favor one over the other. It's probably fair to say that, over a lifetime, people are more often exposed to 5-option items than 7-option items.
Good luck with your work.
What are you going to do with them? For example, if you take the 5 point ones and multiply by 7/5s, of take (x_i - mean(x))/sd(x), these are linear transformations so won't affect things like correlations. However, as the other commentators have said, there are issues that these transformations may not accurately reflect how people use these scales.
- Badges
- Science topic
- Similar topics
- Psychology