Hi Camilla,

I hope you are doing well! I actually want to offer an alternate analysis to determine the unidimensionality of the scale. Instead of using the Kaiser rule (based on eigenvalues) to select the number of components extracted, I want to recommend forcing an extraction of one component and skipping the rotation (because there's only one component). Then, I would see which items do not load onto the single component above .4 and eliminate them.

I would rerun the principal components analysis with the new items using a parallel analysis to determine the appropriate number of components. It is more likely now that the parallel analysis will suggest extracting one component. If it does, then you are done and you have a unidimensional scale.

If the parallel analysis suggests extracting more than one component, then extract the suggested number of components from the parallel analysis and then further eliminate any items with weak loadings or strong cross-loadings. Then you should be left with a unidimensional scale. You can check this scale by doing one more PCA to see that only one component should be extracted.

The reason I am suggesting this method over just eliminating all items that are not loading onto your desired component right away is that you might be able to retain more items, increasing the chances of getting better internal consistency. Of course, if this suggested method does not work for you, you could just eliminate all the weak loadings or items with cross-loadings right away. You might consider eliminating your desire to have a unidimensional scale and have a scale with subscales, which would be more reflective of the data.

https://global.oup.com/us/companion.websites/9780199734177/supplementary/factors/factors_a/

Hope this helps!

Best,

Bryant

In general, I would not recommend PCA when you are trying to assess the number of factors in your data. This is basically a technical issue about the underlying assumptions, but instead I would recommend either Principal Axis Factoring or Maximum Likelihood. In addition, I would recommend a using correlated factors (oblique rotation) because this is a far more realistic assumption than orthogonal factors.

Try that combination see if it helps clarify your cross loadings.

Camila Sneli Mørch, I hope I'm not going to simply repeat some of the good advice you've received above.

First, please appreciate that there is a distinction between principal components analysis (PCA) and exploratory factor analysis (EFA). David Morgan was getting at this, but might not have emphasized it sufficiently for you and others to appreciate that the distinction exists. Many researchers use PCA, probably because so many other researchers do so - but, to me, that's a sheep-following-each-other (or a blind-leading-the-blind) kind of activity.

Second, within EFA, there are a number of extraction methods. The most commonly used of these are principal axis factoring (PAF) and maximum likelihood (ML). I won't go into details here, but PAF is probably the safer one of these two to aim for.

Third, I think that, right at the start, you should try to identify, as accurately as possible, the likely number of factors in your data. As mentioned above, the Kaiser criterion is probably the least effective of these methods. I recommend using parallel analysis, which is no longer difficult because there is at least one easily-used site on the web for conducting it. (Let me know if you'd like more information about that.)

Fourth, and again following up on something from David Morgan's post above, I recommend NOT using varimax rotation unless there is a strong theoretical reason for doing so. As with use of the Kaiser criterion, use of varimax rotations is very common and probably so because researchers just follow in line with what other researchers have done - whether doing so is appropriate or not.

In your case, it certainly seems strange that you are seeking unidimensionality but, by mere dint of using a varimax rotation, you are forcing orthogonality on your data. You are fighting yourself!

Finally, for the moment, I'd recommend that you choose a promax rotation (as one of several oblique rotations) - but that's because, in my experience, it has produced cleaner and more interpretable solutions. That might simply have been because of the nature of the data I was working with, however.

All the best with your research.

If your data follow a normal distribution, I would suggest ML extraction method with promax or direct oblim. Rotation.

The varimax rotation method assumes you are looking for dimensions with minimal correlations.

If the issue still persists, then this item does not in reality belong to the dimension in this sample. You may report this and justify it.

You may also enforce one factor solution and examine the scale's ave and msv to examine its convergent and divergent vidty befire and after the inclusion of the troubled item.

Afterall unbelongining items are troublemakers in subseqent cfa

Thank you, everybody! Great answers that cleared up in a lot of confusion for me.