I have a self-designed 27 item scale that assesses a particular experience. I have run an exploratory factor analysis to identify potential subscales with a promax rotation (I have good reason to believe that the factors are not orthogonal). I end up with a four-factor solution. However, no matter which loading I use as my cutoff (I like to use .5, but I toyed with .3 to .7), I end up with several items that load on more than factor. Using a loading of .5, I have subscales that make sense but, again, some items (about 8-9 of the 27) are used more than once. The reason why it makes sense is because one specfic type of experience (one subscale) can also be a part of a different type of exeprience (another subscale). On the other hand, adding up all of the items to make a single score yields a score with good internal validity (Cronbach's = .77).
My question is this: Is it acceptable to present a scale with items that are used in multiple subscales, or is it better to simply call it one overall trait?
It depends on what you want to use the scale to do. If you want to present the factor analysis results and provide descriptive information about your scale and sub-scales, you can make the sub-scales as they load and discuss the difference in meaning or impact of the individual items that show up in different measures.
However, if you want to use the scale/scales as outcomes of some independent predictor, you will have a problem identifying what aspects are unique to sub-scale 1 compared to sub-scale 4. It can be done, but you will need Structural Equations Modeling to do it, so you can model the overlap between subscales. For this type of analysis, it would make sense to the reader if you start with the full scale, then run SEM on the subscales with their correlations estimated as part of the model.
IMHO
Just to add to Julia's response, if your plan is to market this scale then you might want to remove items with multiple loadings, but if your goal is just to describe the data that you have, then just examine different models. Is there a reason that you think the factors might not have been correlated? Would a bi-factor model be worth considering?
A couple of considerations:
1. Regarding your overall number of retained factors, I would recommend running a parallel analysis (PA) in addition to your more traditional factor analysis (FA). Although results are often similar, PA is now widely recognized as a more accurate procedure (as is MAP; see O'Connor 2000 for more info and links to syntax - O'Connor, B. P. 2000. SPSS and SAS programs for determinign the number of components using parallel analysis and Velicer's MAP test. Behavior Research Methods, Instruments, & Computers, 32, 396-402).
2. It sounds as though you are considering ascribing individual items to multiple subscales for measurement purposes. This is quite atypical and I wouldn't recommend it, as it (as you noted) indicates some bleeding between your factors (which we would expect to be correlated with one another, but not contaminated across factors).
3. That said, it sounds as though you are still in the initial stages of measure development. At this stage, I don't necessarily see cross-loading items as a problem. Part of the scale development process is to do just this - to identify such items and revise or remove them accordingly. Are they unique in some way that you are hesitant to remove them? Or are they getting at something that may be better covered by another (non-cross-loading) item? I would recommend taking such considerations into account.
Best of luck.
Yes. The fact that you are using eigenvalue cut-offs of much less than one means that you are almost certainly over-extracting factors. As Maura says, a parallel analysis is a much better way of determining the correct number of factors. I suggest you read the chapter below (Velicer et al, 2000) to better understand how the number of factors should be determined. Also be mindful that sample sizes of less than 100 can impact on the accuracy of factor analysis results.
An online parallel forms analysis engine is available here (Patil et al., 2007):
http://smishra.faculty.ku.edu/parallelengine.htm
Velicer, W. F., et al. (2000). Chapter 3: Construct explication through factor or component analysis: A review and evaluation of alternative procedures for determinng the number of factors or components. Problems and Solutions in Human Assessment: Honoring Douglas N. Jackson at Seventy. R. D. Goffin and E. Helmes. New York, Springer-Verlag: 41-71.
Patil, Vivek H., Surendra N. Singh, Sanjay Mishra,and D. Todd Donavan (2007), "Parallel Analysis Engine to Aid Determining Number of Factors to Retain [Computer software]. Available from http://smishra.faculty.ku.edu/parallelengine.htm; Utility developed as part of Patil, Vivek H., Surendra N. Singh, Sanjay Mishra, and Todd Donovan (2008), “Efficient Theory Development and Factor Retention Criteria: A Case for Abandoning the ‘Eigenvalue Greater Than One’ Criterion,” Journal of Business Research, 61 (2), 162-170.
I think it can go either ways. The most important things in a scale are the validity and reliability of the items.
It depends on Cronbach's alpha results. Find out the highest value: using the overall or the principal component, that is, the first factor with the highest Cronbach's value. Better to conduct both analysis. Thank you.
- Badges
- Science topic
- Similar topics
- Psychology
- Psychometrics