I want to examine the factor structure of two scales respectively (scale A: 14 items; scale B: 26 items). We are trying several methods: CFA followed by EFA; Rasch analysis and IRT respectively. IRT seems to be a more and more popular methodology for the purpose. But I have also come across some studies using meta-SEM to examine the factor structure. It basically refers to: getting the pooled inter-item correlation matrix from the respective inter-item correlation matrix from a batch of related study, then running SEM on the pooled matrix for testing the factor structure.
For meta-SEM, is item-level intercorrelation is must? Can it be based on intercorrelations among subscales instead of all individual items?
Which method would be relatively more rigorous? Any recommended examples or resources? Thanks a lot!
If you can share about model i.e. variables and concepts, I will able to explain more. Note. first EFA is require for newly established items then CFA can be performed to confirm the items.
I am not quite sure what you mean by examining the factor structure of your two scales. Do you just want to confirm the composition of the scales (=are the items OK), or do you suspect that several latent factors underlie your scales?
In the first case, you should just perform CFA to see if the model (the composition of the scale) fits your data, and the item parameters are satisfactory. If not, there are several possibilities, one of which is to delete some items from the further use of the scale, another to allow correlations between some items (correlated error variance).
In the second case, you could use ESEM (Exploratory Structural Equation Modeling) which is a form of combination of EFA and CFA.
Rasch analysis is a form of IRT (even if they come from different traditions). Rasch analysis presupposes that the loadings of all items on the scales are the same, which seldom is true, but it is possible to test the assumption with CFA. IRT, in fact, is a form of SEM, just developed in another tradition based on categorical data, while CFA (Karl Jöreskog) were developed for continous data. Today it is basically the same thing, and there is software which can handle both types of data, only IRT uses another parameter scaling.
The program Mplus is excellent for all the analyzes, and gives both parameter estimates in the traditional CFA parameterization and in IRT parameterization. (Mplus, and also a lot of learning material, can be found at www.statmodel.com
Best
.
When there is a strong theoretical foundation of the scale, you may not need to perform the EFA. Please verify this first. When this is confirmed (strong theoretical foundation) of the scale, I suggest you to perform a CFA first to examine if there appears a relationship between observed variables and latent construct. This can be done using SEM from AMOS.
Good luck!
First, you perform na EFA and after a CFA. However, it might not be needed to perform EFA. For the confirmatory factor analysis, there are very good books that can guide you step by step, once the theoretical requirements are met.
In order to provide you with better feedback, I would need to know what software will you be using, AMOS?
IRT has benefits but generally assumes unidimensionality of the scale. If you think that your scale will contain more than 1 factor then you may consider some other means to establish the structure of your scale. IRT, however, will permit fewer items and greater precision of those items.
More traditional methods of scale validation include the reflective models of EFA and CFA. Your choice of EFA depends upon your a priori conceptualization of the theoretical orientation of your scale. But, principal axis factoring is often a good choice. Once you conduct your EFA then you should refine your model (i.e., by removing items that have poor loadings (
Evaluating dimensionality with IRT/Rasch is really just secondary, as poor fit of those models just reflects that their assumption of unidimensionality does not fit. To evaluate factor structure of a scale, you need to use item-level factor analysis methods based on tetrachoric correlations. A free program for this is MICROFACT http://www.psych.umn.edu/faculty/waller/downloads.htm. There also used to be TESTFACT which is now part of IRTPRO.
Thank you all researchers for your precious advice and resources! We use SPSS for EFA and mplus for CFA. There are quite clear theoretical foundation of the scales, so we went for CFA at the first step. But later when we collect more samples, we found that the original 3-factor structure was not so stable, but the underlying 6-factor structure performed relatively better (the 2 domains under each factor in the 3-factor structure become an individual factor in the 6-factor structure).
Based on our initial information search, it seems some reasons for considering meta-SEM are: 1) able to more stringently combine many samples after adjusting for factors like sample size and weightings; able to conduct subgroup analyses; and c) able to test an overall factor model even if not every sample contains all the variables in the overall factor model.
And since we are not fully clear about when we should use CFA-EFA, IRT, and/or metaSEM? Or what are the basic differences among these three major techniques? It just seems IRT focuses more on the item-level performance, and based on your advices above, IRT seems to be for unidimensional scale?
IRT explicitly assumes a unidimensional scale, unless you are using multidimensional IRT. That's why those other programs exist, to help evaluate that assumption. I just remembered that there's another one called DIMPACK from Bill Stout at U Illinois.
- Badges
- Science method
- Similar topics
- Mathematics
- Statistics
- Statistical Techniques
- Structural Equation Modeling