You can re-score each variable, x, as its z-score, where z = [x - (mean of x)]/(sd of x). That will give every z-scored variable a mean of zero and a std. dev. of 1.0. With equal dispersions, all variables will have the same weight in an additive composite scale calculated from the z-scores. This approach is covered in most standard textbooks on forming composite scales, but I don't have access just now to my usual resources so can't give you a citation; sorry.

Z-scores treat the underlying variables as interval/ratio in measurement, whereas Likert items are actually just ordinal. Doing EFA on those items makes the same assumption about level of measurement, so you may not be bothered by that aspect of the z-score approach. But if that is a concern, you can accomplish approximately equal weights by just re-scaling your variables to give them all the same range. You could re-code your 4-point items as follows: 1=1, 2=2.33, 3=3.66, 4=5. That would (approximately) preserve the equal-appearing intervals of your original 4-point items, while giving all items the same range, therefore approximately the same dispersion, therefore approximately equal weight in a composite, additive scale.

I would also recommend to test for the internal consistency of the items that compose the scale (Cronbach's alpha).

Disclaimer: I am not an expert factor analyst. However, I do believe it is fairly conventional to use a matrix of polychoric correlations as input when factor-analyzing ordinal variables. If you do a search on , you'll likely find an example using the software package of your choice. HTH.

Thanks for your answers, everyone! I have tested the reliability and it's close to 0.90 (using Cronbach's alpha). I will apply the approach suggested by Burke Grandjean and re-scale the 4-point items. Most of the items are negatively skewed so I fear that using the Z-score approach will somehow distort the data, although I am not certain. Of course, I want to use the method that is the most precise (to get the closest estimate of the value in the population). A low score on the items means a higher degree of marginalized students, so by re-scaling the 4-point items the only danger, as I see it, is that the composite measure could potentially underestimate the amount of marginalized students.

I found this video on YouTube by Dr. Todd Grande on how to calculate z-scores in SPSS.

https://www.youtube.com/watch?v=eblFi4xUaSU

I will use that method and divide the resulting composite variable into four categories and compare the results.

Martin Brygger Andersen wrote: "Most of the items are negatively skewed so I fear that using the Z-score approach will somehow distort the data, although I am not certain."

The z-score transformation does not change the shape of the distribution, if that is what you fear. It simply re-expresses each score as a deviation from the mean in standard deviation units.

Two follow up comments, based on the additional info you've now provided:

In SPSS (and some other software packages), the Reliability procedure will give you a value called "Standardized Item Alpha." That is the alpha you would get for the composite if you z-scored every item. If that value is greater than the regular alpha for the same composite, then the implicit weighting of the unstandardized (non z-scored, non re-scaled) items is reducing the reliability of the composite. If the standardized item alpha is less than the regular alpha, then the implicit weighting is actually increasing the reliability of the composite (probably because one or more of the 5-point items are, individually, more reliable than one or more of the 4-point items). It might be informative to compare three values of alpha: the standardized item alpha, the regular alpha on the non re-scaled items, and the regular alpha you get after re-scaling the 4-point items to have a range of 1 to 5. My guess is it probably doesn't make much difference in composite reliability, as estimated by alpha, which of the 3 approaches you use to form the composite. In my experience, it rarely makes much difference unless some items have very different scales from others (say, some 4-point scales and some 7-point or even 10-point scales), or one or more of the 4-point items have hugely different (either higher or lower) item reliability than some of the 5-point items.

Also, I don't see much benefit in your plan to "divide the resulting composite variable into four categories." One of the main advantages of forming a composite scale is that it comes much closer to being a continuous, interval-level measurement than any of the individual, ordinal-level items. Collapsing the composite back into just 4 categories discards that advantage. Instead, I recommend using the uncollapsed composite in your analysis. If you need to designate a cut-off point, below which you regard the students as marginalized, you can do that by setting an arbitrary score (say, an average score of less than 2.0, or 2.5, across the items in the composite, or more than one standard deviation below the sample mean of the composite, or 1.5 sd below, or 2 sd, or whatever). Or you could examine the frequency distribution of the composite and look for a natural break (a gap in the scores where there are few or no cases) near the low end, and designate that as your cut-off to define marginalized.

Hi Burke,

Thanks for that response. That made perfect sense and it was immensely helpful. I have tried the different approaches in SPSS, and the difference in reliability was quite small, as you had anticipated.

1) I applied reliability analysis on the 7 items (non z-scored, non re-scaled). The regular alpha was 0.833 and the standardized alpha was 0.836.

2) I applied reliability analysis on the 7 items (3 were re-scaled from 4-point to 5-point items). The regular alpha value increased to 0.835, the standardized was still 0.836.

3) Then I standardized all items (mean = 0 and SD = 1) and the alpha value was 0.836 and the standardized alpha was 0.836 (of course).

Based on these three methods, the last method seems to produce the most reliable composite measure. However, I want to keep the original 5-point scale (using non-standardized items) as this is easier to interpret. Thus, I have applied the second approach seen above. But it seems, based on these results, that all three methods will yield similar results. I plan to divide the variable into 4 (or 8) categories in the first part of the analysis but in most of the analysis I will use the composite measure as a dependent variable in ANOVA (as a quantitative or continuous variable).

Your suggestion of using a cut-off point is extremely interesting. Yet, based on theory, I want to avoid a simple binary interpretation (non-marginalized vs. marginalized) for now and instead consider marginalization a continuum or a process. However, I might use that approach later and compare the results.

Thanks Bruce, I completely agree. I want to avoid using data transformation tools that change the variable distribution. Thanks for the clarification.

Martin Brygger Andersen and other contributors to this thread,

In my view, there are some helpful ideas and responses up above. At the risk of throwing the cat among the pigeons, and of being plain wrong, I'd like to make the following suggestions for you, Martin:

Because a neutral response doesn't really clarify the "structure" of opinions about a topic, I wonder whether you could initially convert the responses of neutral to missing data. Then, recode the options of 4 and 5 on the 5-option scale to 3 and 4, respectively, and do your analyses combining responses from the two scales.

If you did that in EFA with SPSS, I think you should choose the "Exclude cases pairwise" under Options so that you didn't lose data because of missing responses created by discarding neutral responses.

Apart from that, with as few items as you have, a coefficient alpha value of .90 strikes me as an indication of high redundancy among your items. Might that be the case?

And, yes, Martin, I recommend avoiding transformations if you possibly can. In my experience, they seldom have much effect on the shape of a distribution and they make interpretation massively more difficult.

All the best with your research.

Robert Trevethan makes some interesting suggestions that could well be worth exploring. I'd advise caution, however. Usually, providing an explicit Neutral category picks up not only people who would otherwise decline to answer, but also many who would otherwise pick one of the Somewhat categories. So I suspect putting all of the neutrals into missing data will greatly reduce the number of valid responses on that item. And pairwise deletion of missing data can yield a correlation matrix which is not positive-definite, which makes a solution for the factor structure either impossible or unstable.

If you do entertain the suggestion of re-defining a Neutral response as missing, you could also explore the opposite idea: redefine all missing responses to Neutral and include them in the analysis. That would put the 4-point items onto a 5-point scale, with 1=1, 2=2, 3="neutral/missing," 4=the old 3, 5=the old 4. The 5-point items would remain on their original 1-5 scale, but with no missing data since the no-answer responses would be assigned a score of 3. Under that approach, the issue of pairwise vs. listwise deletion of missing data goes away, as there are no cases defined as missing.

In general, I'm not a fan of assuming a no-answer response is equivalent to an explicit neutral response, whether as Robert suggests or in this opposite fashion. Those two responses are cognitively different for the respondents, in my view. However, if you want to spend some time exploring these alternative approaches, you might get some interesting results.

Burke D. Grandjean, thanks for taking the effort to contribute your ideas above. I guess that some "testing" of the data would be in order. Furthermore, if the neutral response option was used infrequently, there might be little problem.

Your redefining of missing to neutral, rather than the other way around, is an interesting prospect. It hadn't occurred to me.

I guess the most important things are to plan surveys / research carefully from the start and, if problems occur, to do what is maximally honest and indicative of participants' intentions when they responded to a survey.

Martin Brygger Andersen, if you had the energy and inclination, I (for one) would be very interested in how you solved your problem. It's quite a delicious one. :-)

Also, I'd be interested to know how much the neutral category had been used by your participants.

The kind of problem(s) you face crop up every so often, so it's interesting to see how different ones of us addressed them, and why.

Hello Trevethan and Burke,

Thanks for your thoughtful suggestions.

Regarding redundancy, I have checked the phrasing of all items, and none of them carry a completely identical meaning. Devellis (2016) only really considers redundancy a problem if the items carry a similar meaning and only differ in a grammatical sense. So I think that an alpha value of 0.84 is alright.

I have tested different recoding strategies, and, to my surprise, the results were actually quite similar. I tried recoding all 4-point items as Burke suggested: 1=1, 2=2, 3="neutral/missing", 4=the old 3, 5=the old 4. The results were almost the same as before. Perhaps because I have about 40,000 respondents with a response rate around 94%. So there aren't that many missing values in the data.

Also, it makes sense to me to alter the data as little as possible - to keep the analyzed data as close to the original data as possible. Thus, I prefer to recode/rescale the 4-point items instead of the 5-point items.

Martin Brygger Andersen You can consider using partial least squares structural equation modelling (PLS-SEM or PLS-PM).

Dear Burke D. Grandjean

Thank you for these helpful suggestions last year. I am close to handing in the final version of this article but the reviewers want to know whether I have any reference for this specific re-scaling method that you wrote about.

"You could re-code your 4-point items as follows: 1=1, 2=2.33, 3=3.66, 4=5. That would (approximately) preserve the equal-appearing intervals of your original 4-point items, while giving all items the same range, therefore approximately the same dispersion, therefore approximately equal weight in a composite, additive scale."

Do you have any reference for this specific method? Or is it merely based on mathematical logic? Do you know where I can look for a discussion on the different re-scaling methods that you suggested?

Have a great summer and take care.

It's based on math logic, and on empirical results such as you report for the effect on alpha of the different scaling approaches. More generally, I guess you could consult works on covariance algebra, in particular the theorem showing that the variance of a sum is the sum of the variances (adjusted for covariances in the case of non-independence); hence the variance of a composite scale will give equal weight to the items if the items themselves have equal variances. General works on scale construction might also mention this. Unfortunately, I can't point you to a specific reference. I hope this is, at least somewhat, helpful.

Burke D. Grandjean

Thank you. I can use use the refence to Field (2018) for the variance sum law. It makes sense. Fortunately the items do have equal variance and should therefore should be weighted equally in the composite scale.