When it comes to dichotomous and ordinal item data, the Item Response Theory (IRT) family of models provides very good tools for fitting models and estimating factor scores. There is a lot of IRT software around, also packages in R.

Best approach is to construct a composite index by using Factor Analysis approach of Principal Component Analysis

When it comes to dichotomous and ordinal item data, the Item Response Theory (IRT) family of models provides very good tools for fitting models and estimating factor scores. There is a lot of IRT software around, also packages in R.

Software such as R, Mplus, LISREL etc can fit latent factors to categorical data quite well. It's typically viewed as an item response theory approach where, if items are dichotomous, you have a traditional Rasch, 1pl, or 2pl model, and if it's Likert scaling you have a graded response model. See the examples for fitting the model in Mplus here: http://statmodel.com/download/usersguide/Chapter5.pdf

Keep in mind that any "optimal" scores derived from a single sample are likely to be non-optimal in other samples.

The requirements that you state are a pretty classic description of what IRT does. However, there are lots of IRT models to choose from so you will need to be aware of what each is best suited to. The other thing to consider is that while the number correct may not be 'optimal', it often comes out very highly correlated with IRT scores, and is even the 'sufficient statistic' for producing Rasch scaled scores. It may not be as sub-optimal as you think. As well as the various programs mentioned, there is also some limited IRT capability in Factor (http://psico.fcep.urv.es/utilitats/factor/) and in JMetrik (http://www.itemanalysis.com/), both of which are free downloads.

Hello there,

I have had exactly the same problem when doing factor analysis to find latent factors, but all the covariates I put in are not nice, Likert ordinal scales of 1-5 or 1-7. Instead, I'll have some 2-level (dichotomous), 3-level (trichotomous), mixed in with some Likert 5-level or 7-level variables, and I have the same confusion you do when I get to this point! Often the variables are mixes of measurements (such as survey questions mixed with whether or not a person was medication adherent). This is the challenge of using factor analysis to really figure things out - it's hard!

At this point where you are, I step back and actually look at my factors and their ingredients. To get specific, let's say I was trying to identify some latent factors from a hodge-podge of health related data (self-reported health status, claims data, etc.). Let's say I found one - it seemed that everyone 1) visiting a certain location who 2) reported their health as "fair or poor" and 3) took a glucose test at each of their visits was more likely to cost more money over time, and everyone who did the opposite of that cost less over time. When I look at the data, I might say, "Oh, those are diabetics who are not following up on their recommendations". This would help me empirically have ideas of how to weight these 3 variables into a composite variable. Maybe you don't even need a score - maybe this is your finding: If you have a patient who has these three characteristics, do more intensive case management!

Once you decide (or at least can hypothesize) what is going on with your factors empirically, you can test different ways of scoring and see if they make sense with your findings. This a wonderful step to do when you publish, because others can not only benefit from your research models but also can try your approach on their data (a shoutout to Jeffrey Stanton above who points out this should be repeated so we can all discuss what is "optimal").

I hope this helps and I understood your question right. Good luck on your project!

-Monika

Yes, there is such a method, it is part of my generic reference model / approach to classroom assessment, where teachers and lecturers all over the world struggle with the same dilemma of EITHER falsely treating ordinal data as if it where interval or - worse - ratio data OR being forced to learn and apply highly sophisticated models which require substantial mathematical-statistical background and corresponding software packages (see previous commentaries) to use it reliably and validly.

Without denying the merits and usefulness of the latter models/software for certain kinds of research and applications, it occurred to me that there must be better and definitely simpler ways to support lecturers and teachers in their day-to-day handling of essentially ordinal data in a non-research context, without giving up the ideals of soundness, consistency and generality.

Briefly, the method/technique that I developed is based on the concept of lexicographic orderings - which everybody who uses e.g. a dictionary is familiar with - of so-called performance profiles. You may think of a performance profile as a kind of histogram on a quality scale, e.g. 5-point Likert scale (but instead of 5, any number of points is allowed). With a little bit of elementary mathematics it is possible to assign a unique number between 0 and 1 to such a profile which tells you the rank order position of that profile among all possible profiles, so again an ordinal measure!

That's all. The basic formula is - after all - so simple, that it can be readily implemented in Excel and used whenever you have to evaluate multiple ordinal data to produce an overall (ordinal) score. It works fine, and I use it myself since a couple of years in my own courses whenever I have to judge complex assignments which escape the simple false/correct scenario.

If you are interested, have a look at www.passorfail.de.vu for all background that I left untouched here as well as for the complete model. If you want to give it a try and need more support, don't hesitate to contact me.

Good Luck!!

PHV

You can run a structural equation modeling (SEM) where you do not have to worry about the type of manifest (observed) variables (ordinal or dichotomous) that together define the latent variables. Such modeling is available through PROC CALIS in SAS. I do not advocate SAS but that is what I used lately for such analysis. The output would confirm the factor structure and assess the model fit.

Robert,

In order to be assured that the summation of item raw scores can be justified, I would first conduct psychometric analysis based on the Rasch model. It is then also possible to detect potential Differential Item Functioning (DIF), i.e. to rule out if the measure works invariantly across different groups that are to be compared statistically. The good news is that there are several good programs that can be used, and that they all produce an outcome variable with the unit of a logit, i.e. with equal distances across the composite measure, given that the items fit the Rasch model. I use the RUMM2030 software, but there are others, and some of them are eve for free. If interested, have a look at:

http://www.rasch.org/software.htm

http://www.rummlab.com.au/

Dear Paul Vossen, I tried to visit your website but the web-filter blocks it because the website seems to include "sex" contents.

I am curious about your approach and I would like to read something about it.

Please write some references.

Thank you very much!

Please try again with http://pass-companion.weebly.com/, probably it's the forwarding function that the web-filter doesn't like. If it is still blocked, we have to find another solution. Regards, PHV

Now it works!

Thanks!

To score latent factors, I agree with using IRT models (best, Rasch derived models for ordinal data), but only when the ordinal or dichotompus items are statistically independent each one. The only problem in classical IRT models is to analyse responses to each item without consider the whole psychological (or economical) continuum from where such item was drawn. Another way is to use factorial methods, but almost all of them presume that variables are numerical (merely quantitative, nothing ordinal or binary) because both common factors and errors (unique factors) must be asymptoticallu distributed like a Gaussian. The best way, as I experienced in several studies, seems to use Optimal Scaling methods to find Cathegorical Principal Components (DE LEEUW J., 1984 Canonical Analysis of categorical data, 2nd ed., DSWO Press, Leiden - NL;

DE LEEUW J., MEULMAN J.J., 1986 “Principal component analysis and re-stricted multidimensional scaling”, In: W. Gaul and M. Schader eds., Classification as a tool of research, Amsterdam - NL: 83-96) and then apply to such optimally scaled variables an Explorative Factor Analysis or a SEM procedure (Lisrel or other). I use CATPCA Algorithm, included in SPSS by license of Data Theory Scaling System Group (DTSS), Faculty of Social and Behavioral Sciences, Leiden University (NL).

I absolutely agree with previous answers. IRT is definitely one appropriate approach. However, as was said before, you can also estimate a SEM model in the Mplus software using the WLSMV estimator or in R using the lavaan package.

OK, Mr. Brooks, now you've got your 10+ answers and recommendations how to solve your problem, i.e. by what 'optimal' method or software package to calculate 'scores' for 'latent' variables based on 'ordinal' items.

Ten people, ten distinct answers. Well, IRT and FA e.g. PCA featured in several answers. This was to be expected. Why? Because if you don't mention some substantive theory about the domain you are interested in or phrase your problem in a purely technical way, people tend to respond in technical terms and refer to whatever is the currently accepted wisdom in applied statistics and measurement theory. And FA and IRT are undoubtedly very popular, although insiders know and may acknowledge, that there are assumptions and prerequisites for their applicability (history repeats itself, here).

But given that you are asking for the OPTIMAL solution to your research problem, I guess you will be a little bit confused by so many DIFFERENT proposals, especially if you count the number of mentioned software packages. But that's the price you have to pay for a rather abstract, technical question.

For us it would be instructive to learn how you have made your choice, and why so, after considering and trying out the diverse suggestions.

P.S.: Of course, adequate answers (from your perspective) also depend upon what you mean with 'best way'. Should it be as simple as possible? Should it be as fast as possible? Should it be immune against outliers? Should it be grounded in some empirical theory? Should it satisfy important psychometric principles? Should it have some nice statistical properties? Etc.

Imagine what could happen with so many alternative 'best ways'. Let's assume that you can quickly discard 3 of 10 'best ways' for obvious reasons (in your case). From the remaining 7, after more careful analysis, 4 'best ways' seem the involve too much work (for you) to follow up without any clear added value in comparison to the other remaining 3. Let's discard those 4 too. Now there remain 3 candidates, which look pretty promising, at least better in several ways (you decide) than what you had before. OK, you run all 3 analyses on your data, to be sure. What happens? All three ' best ways' produce fabulous results, but ... unfortunately completely different i.e. incompatible results, so you have to accept either one or another 'best way'. However, this doesn't sound like 'honest science', because now you would have to reject *afterwards* an equally 'best way' on unspecified grounds, which opens the door for a lot of subjectivity in your research design c.q. conclusions. Thus it is better to be as specific as possible *in advance* about what counts for you as a good way and what not. Of course, you may be 'lucky' and find the same results for all three 'best ways', but I wouldn't bet on that (worse case approach) and - worse - that doesn't prove anything at all..

I would also (with Francesco) suggest CatPCA - available in Optimal scaling add-on to SPSS. I've got three 3-point questions I want to turn into a dependent variable. I can say yes the items are ordinal in which case the total dimension scores that it saves are pretty discrete. There's an improvement if I say it's multiple nominal in that the scores generated are a little more continuous. Just off to see whether the multiple nominal scores work well in Smart-PLS! (after a more conventional scale construction with 30+ items on 6 point scales).

Hi. CatPCA would also be my choice. As it minimizes the loss, and the loss function is separable by variable, you can define, for each input variable, the desired level of measurement: ordinal for likert-type items, multiple nominal for dichotomous items (and other nominal items), even numerical for intrinsic numeric data that must be linearly transformed. For non-linear transformations of numeric data you can also chose spline transformations (spline ordinal for non linear monotonous and spline nominal for non monotonous transformations). Pay attention to coding, as CatPCA only accepts as input positive integers (variables containing real numbers will be automatically internally recoded in categories, unless you specify otherwise; for binary 0-1 variables you should either recode them beforehand or use the "discretize" option.

Thank you to everyone who has answered and provided links to programs

I have been struggling to get my head around IRT and R in recent times, I am going to have to keep going

I am deep in the middle of a complex analysis (Mulitlevel, Multigroup, SEM) and this is one small part, so thank you again for your help

Dr Robert Brooks