I'm using IRT for a DIF analysis by gender. However it occurs to me that it might be possible to do DIF in a Mokken scale analysis framework. Is there a standard protocol for doing this?
There is a way to investigate DIF in Mokken, but it is not as simple as in IRT models. In Mokken you can fix the order of the item steps to see if they are in the same order in all groups. In MSP5Win, you can get a group difference test. In R, you can calculate the same results when you follow the instruction in the latest manual. I have used the procdeure in 'Apotheken door Clienten Bekeken: schaalanalyses en profielscores 2009' (tables page 11) to test for six variables on DIF. The article is in Dutch and unpublished as the method was replaced with the CQI. The itemstep order are calculated for the total group, but if you take the item step order for a specific subgroup no DIF in all variables can be show (See Grafiek 3 and 4). The very risky variable in the tables is 'ontevreden' (dissatisfied with pharmacist service and having a complaint). I expected this variable could potentially distroy the scale all together. The group having a complaint, could have a very different use of the categories and items all together. The variable did not. I had some DIF, but the critical value kept below 80 (see MSP manual). Hope this helps. Hope you have MSP5Win, in R it is harder to do.
Thanks for your answer, that's helpful to know it's do-able. Unfortunately I have neither MSP5 or the manual. Is the MSP5 manual available online? Or would you have any pointers re relevant R code?
There is no MSP5 manual online. Some information may be found in publications by Ivo Molenaar and Klaas Sijtsma. The program might still be available with Sience Plus Group, but not sure.
The program still works at my university and I can still use it, but I need to do it with out manual. I have work with MSP5 a lot so I can do it 'blindly', but have to think about it. However, in the manual the reason for the critical value is explained and I don't know it by heart. I will ask around where I work. Ivo Molenaar has been my mentor on using MSP5. He had asked me to test the program before it was launched. I had also work with MSP3.
If you would send me your data, I could help you analyze your data and sending the information back to you. I guarantee scientific integrity! If you have a Mokken scale, running the data for DIF analysis would not take much time. Setting up the data file in ASCII is most of the work.
DIF can be explored for a maximum of ten groups. van Ark has a publication in JSS about Mokken that you could use and it give pointers how to do it, but it takes a lot more time and you might have to calculate the test statistics yourself. In MSP5 the tables are given for each DIF test for each variable.
Thanks, but I can't send you clinical data and I was rather hoping to evaluate the procedure rather than have it done for me. I'll have a look at the van Ark publication, thanks for the tip. I guess this might lead to a useful research note somewhere.
The program is still on sale: https://www.scienceplus.nl/msp-software. It costs a little less of GBP 200. It does not mention DIF analysis, but it can do it. The output is ascii and hard to work with (turn into nice publication tables may take some time). I also use the R Mokken program from within SPSS and it will create nices tables for me using SPSSpivottables. (I have integrated SPSS 23 and SPSS 25 with R).
I also make syntaxes to run analysis under R alone. Sometimes one needs to check if a newer version of R still does things the same way as a previous version. For instance for Mokken the search needed seach, as term, but now it uses iasp (For example: "test30
@John. When studying IDF in Mokken one need to fix the steporder in one specific way. When you make a scale, MSP will give the steporder found in the analysis that define your final scale. In MSP you can set this, in Mokken in R I have not seen if this is possible. In my study ("Apotheken door Clienten Bekeken") I have used the order from one analysis to be tested in for all other DIF analysis. If you have missing values for a variable you want to test, it will recalculate the steporder based on the non-missing values you will test in the analysis. For instance you could have five groups with SES and one code is for missing values, if you delete the missing values group the steporder will be based on the cases from the remaining groups. This is a big difference with dichotomous or polytomous RASCH models. The order should be sample independent and any subgroup could be tested against any other.
This morning I heard from Abel Keun a new version of MSP will appear later this year. MSP6 will run on all Windows platforms. It will also be able to read SPSS directly.