There are many different programs for conducting IRT, some are stand-alone products and some are components within larger statistical programs. What are the most popular programs in common use?
Additionally, does anyone know of any independent comparisons of the various programs to ensure that they produce identical results?
I prefer open source R with package mirt, but there are a number of different IRT packages in R. Popular closed source programs are parscale and the more recent irtpro. i think mplus might have irt capabilities, but i am not sure.
dont know about comparisons between programs. best, felix
Hi Ariel!
It depends on whether you have odd things to do. If you simply have 20 binary items and 200 students, and want a Rasch model any of the packages should have that (even general ones like SYSTAT, and I even went to talk last week on STATA where they have built a GUI but haven't made it part of any of their implementations yet). I agree with Felix that the R package mirt is very good (and others are described http://ftp.uni-bayreuth.de/math/statlib/R/CRAN/src/contrib/Views/Psychometrics.html). For $ packages, the IRT suite Felix mentions are at http://www.ssicentral.com/irt/ and are state-of-the-art. Before there were a lot of the R packages for this, I used them (see the attached ... note I was in the UK then so use Brit terminology).
Dan
Article A dialogue about MCQs, reliability, and item response modelling
Thanks Felix and Daniel!
FYI, Stata just came out (yesterday, in fact) with a version 14 that has a suite of IRT commands.
I have heard some people rave about Winsteps, and others about the packages in R, so this got me to thinking about the various software implementations and their comparability.
I am hoping that this query will generate a sufficient number of responses so that we can see the rankings, and of course, get exposed to various programs that we previously did not know about!
Ariel
Does the new STATA have a GUI for them? The speaker was talking last week (well, two weeks ago) as if they were just using the GUI internally at this point.
Oh the IRT suite is nifty, and it comes with its own manual. Check out the attached link.
So I wonder how this compares to other packages (and how do the other packages compare), and of course most importantly, do we always get the same results.
http://www.stata.com/stata14/irt/
Wow, that is the screen we were being shown and told it wasn't up yet. Hmmm.
I'm not a STATA person and I don't think anyone here is (mostly R and SAS). Hopefully they all should come close if the options are all set the right ways. The IRT I am working on now involves incorporating the response times.
The University of Massachusetts has a somewhat dated, comprehensive list of IRT software and what their strengths and weaknesses are (http://www.umass.edu/remp/software/CEA-652.ZH-IRTSoftware.pdf).
I don't know of any studies that have compared commercial software for unidimensional models off hand, but Chris Han did a study last year in Applied Psychological Measurement comparing commercial software for multidimensional IRT packages if that is of any use (http://apm.sagepub.com/content/38/6/486).
SAS also has in experimental PROC IRT procedure that is new in version 9.4 although I haven't used it and don't know much about its capabilities.
Ariel, yes, the individual item response times. There are a few models out there (van der Linden's hierarachical model, Q-diffusion by van der Maas and colleagues, standard Ratcliff diffusion, etc.). I'm co-author on a paper for NCME in Chicago in a few weeks (I want be there, but). We will write up hopefully soon after. (Hi Dan!)
The R package "ltm" works with IRT models, and is easy to use. This paper shows ltm in use:
http://www.jstatsoft.org/v17/i05/paper
I was comparing speed for mirt and ltm, and found mirt to be much faster.
SAS 9.4 has added Proc IRT. I have used it for one problem and thought is was very intuitive. If you are comfortable with factor analysis is SAS you will find the look and feel very familiar.
O compared results of Parscale 4 and R package ltm - they are almost identical. Also recently came across quite good jmetrik program (see http://www.jmetrik.com/). They also published a manual
Ariel, IRT programs produce estimates, never the true values of the parameters. IRT programs that use different estimation methods produce different results. Some IRT programs include several options that also produce different results. Also IRT programs use iterative estimation techniques,so the final values of the estimates depend on the stopping rules for the iterative process.
The R Project for Statistical Computing or LISREL statistical software package are highly recommended!
http://www.uv.es/innovamide/l4u/R/R.wiki
There are so many different R packages, SAS procs and scripts out there, and special purpose software packages as well.
For options that work also when using very large databases, I put together a presentation that was given at an OECD Workshop in international assessment:
https://www.oecd.org/skills/piaac/events/Matthias_von_Davier_IRT_software.pptx
- Badges
- Science topic
- Similar topics
- Geoscience
- Cartography
- Mapping
- Maps