I'm exploring the concept that scales can be ordered and that certain items should carry more weight in a scale. I came across guttman scalograms and Mokken scaling techniques. Creating the initial Mokken scale makes sense. What I don’t get is that after I get my H coefficients and AISP to run with Mokken, how do I evaluate the data in a meaningful way?
If I use a mokken analysis on a 10 item likert survey, it wouldn't make sense to get an overall composite mean score to represent the latent ability since I established that items are ordered on difficulty. Do the H coefficients determine item weights? How can I sum a participants score on my newly created Mokken scale?
You might use the mokken scaling to get a unidimensionality and monotone properties in the initial work. But you might need to use Rasch-family models to getting thresholds based true-scores.
In regards to IRT, I would suggest you to start non-parametric IRT (KernsmoothIRT) in R to evaluate/describe the use of response options for all items. May want to drop some if they do not work well with overall scores. If response options look OK, move to a parametric graded response model (ltm package in R) for item parameter estimation, identification of patterns of responding, and constructing of a standard error of measurement that accounts for where on the continuum of scale performs best/worst. It might be meaningful to start with the theory behind the scale development to help.
In Mokken and Rasch the sumscore should be a sufficient/nessessary condition. Which means if you really have a Mokken scale, you do not need to weigh. However this only holds when the double monotone model (DM) for Mokken holds. Did you use MSP5 to analyse the data or did you use 'Mokken' inside R? In MSP5 you can obtain critical 'crit' values for the scores for the P++ and P matrix. In Mokken in R (see van der Ark, Mokken Scale Analysis in R in JSS) page 11 and 15. MSP5 calculate a summary test statistics 'crit' to see if the violations are troublesome. Using DM one can use the sumscore, but its level will be ordinal. The Rasch model will upgrade the ordinal data to interval, but the strong assumptions of the Rasch model must be met.
Thanks everyone for your responses. I agree that it probably makes sense to start off with the Mokken package in R to determine the initial scale properties and then switching to the LTM package in R in order to run a Graded Response Model IRT.
As for Peter's comment, I am using the mokken package. To your point, can we also use sum score for polytomous data as long as it adheres to the DM model?
Mokken can be used for both 0/1 as polytomous items without any problems. If the DM model holds you can, but when I really want to be sure, I will also investigate DIF within the Mokken framework.
- Badges
- Science topic
- Similar topics
- Mathematics
- Statistics