I have a multi-part question that is a little ill-formed but the gist of it is this: I have 15 or so dichotomous items that divide into drug use behaviors that put one at risk for an overdose and protective behaviors that potentially reduce risk. I also have 3 socioeconomic indicators - also dichotomous.
My goal is to analyze these variables to obtain a scale that assesses risk for an overdose. I do have past-year overdoses experienced against which I could use any scale to determine predictive validity (sensitivity, specificity).
Prior analyses I did on the risk items only indicated they could be represented by two factors. Preliminary analyses that included the protective items found, not surprisingly, they formed a third factor.
So it seems I have a preliminary "scale" that includes 3 subscales. And I am stuck at this point with a few questions about it:
1) I have read some work and watched videos about assessing the structure of multiple subscale scales that leads to assessing bifactor or higher order factor structure. It's interesting but does that kind of analyses really add anything to scale construction as opposed to conceptual/theoretical clarity?
2) Also read about the use of multidimensional item response theory (mIRT) (as opposed to factor analyses as a means of scale development. I appreciate the more detailed information IRT provides and scale items but again, is there any advantage to using mIRT for scale development versus EFA/CFA? And even so, what is the path from mIRT to setting up an actual scale, determining how to score the items and then testing the predictive validity of the scale and scoring?
Any insights would be much appreciated.
Hi James A Swartz ;
If you know the structure of the scale, you should use CFA. If there is a correlation between the 3 factors (r > 0.40 for example), then you can conduct a second order CFA.
MIRT is actually used in scales known to be multidimensional and its purpose is to obtain ability estimates of individuals. If your aim is to examine the structure of the scale, then you should use CFA. MIRT does not provide as much information about the structure of the scale as CFA. You can examine item statistics, but items that are not suitable for MIRT will not be suitable in CFA. Second order
CFA gives you the opportunity to examine whether a total score (or average score) can be obtained from the scale.. In other words, if you get a model-data fit as a result of second order CFA, you can use the scale as unidimensional and you can also use the sub-dimensions separately.
In this respect, second order CFA gives you extra information.
Bests.
That is extremely helpful. Thank you very much for the quick answer. Makes a lot of sense that the second order CFA would provide information on being able to use it as a single scale.
Any recommendations for how to score items? To be useful, I would think a scale that just summed the items would be best. Applying weighted items would be too complex in my estimation. Any thoughts in that regard?
You can use total score or average score.
You can calculate the sum of all items for the total score. For the average score, the calculation is made by dividing the total score by the number of items.
I have analysed the relationships between total score, factor score and IRT ability estimations. You can take a look my article below.
Of course, these results are valid for unidimensional structures, but they may also give an idea for your situation.
Article Can Factor Scores be Used Instead of Total Score and Ability...
- Badges
- Science topic
- Similar topics
- Geoscience
- Cartography
- Mapping