Hi,
I analyzed my data with two different IRT tools (STATA and XCalibre 4) and got totally different outputs.
For STATA, discriminability parameters for most items are negative but for XCalibre 4, they are all positive. Of course, item characteristic curves for each item also look very different. I can understand there could be small differences between the tools, but this is too huge which cannot be acceptable.. Can anyone help me figure out this problem?
Thank you in advance!
P.S: Can it be due to small sample size? (Sample size is only about 50..)
Check the df. If they are different, you are fitting a different model with each software. If df are the same. Attempt to provide the same starting values and see.if they end up in the same place. But...... you're right .... a sample of size 50 is insufficient. Count how many parameters you're estimating... I would not be surprised if you have more parameters than observations. For the mosels you're fitting, Irt is nothing more than multivariate logistic regression where the predictor is unobserved . How many observations would be confident with to run a single logistic regression with one predictor? Multiply that number by the number of items you have..
Thats the sample size you need for your irt model
Without knowing which of Stata's many IRT models you ran, or anything about your data, the best advice is that you probably got different output because you ran different models.
Can you tell us a little more about what exactly you did?
Thank you Ronán.
My data is binary— yes: 1, no: 0, and has 17 items to test. I think I ran XClaibre with MLE and STATA with log likelihood. Can it explain the differences?
What model did you run? Stata runs 8 different models as a basic. I don't know about XCalibre. Did you use the dialog in Stata? If so, what options did you select?
Hi....look at estimation method in each program....may be they different...
Hi Ronán,
I used the dialog in STATA and 2PL model for analysis. I did not change any advanced options. Can we set theta estimation method in Stata?
Hi Waleed,
I used Maximum Likelihood (MLE) method for XCalibre 4 but I’m not sure which estimation method I used in Stata as there is no tab for estimation. But I guess it also used MLE..
- Badges
- Science topic
- Similar topics
- Mathematics
- Statistics
- Sampling Design
- Sample Size