You wrote: "The theta parameter (latent variable) in Item Response Theory (IRT) is not a measure for test-takers' ability although it may be positively correlated to the test-takers' ability."

Can you define what you mean by "measure?" Otherwise, it remains unclear what you are concerned about.

Also, I cannot follow your analogy (area of a rectangle). Perhaps you can elaborate a bit. The latent trait variable ("theta") in IRT is not equal to (and is not defined as being equal to) the sum of observed item scores if that's what you are referring to (although if a 1-pl [Rasch] model is correct for the data, the unweighted sum of item scores is often a close approximation/highly correlated with the Rasch person parameters/estimated theta scores).

Hi, Christian,

The concept of measure is defined in Measure Theory. The key concept of a measure is called Additivity. The latent variable in IRT is not a measure because it has no Additivity. In the example of rectangle, the product of a rectangle length and width is additive in terms of the area of rectangle while summation of a rectangle length and width is not (but summation of rectangle length and width is additive in terms of a rectangle perimeter).

Any measure must have its decomposition theory such as Probability and Entropy etc. (product of rectangle length and width does have its decomposition expression in terms of rectangle area). It is known that IRT has no its decomposition theory. Without the decomposition expression, IRT presents many serious problems such as (1) paradoxical issue for both IRT and NIRT (should be MIRT -- Multi-dimensional IRT); (2) In MIRT, there are only abilities "measures" associated with subscales, but no ability measure (overall score) associated with all testing items although the ability "measure" in one-dimensional IRT is associated with all testing items, but not consistent with those "measures" from MIRT; (3) IRT is not consistent with CTT (classic test Theory). By the way, the total score (number of correctly responded dichotomous items) is a special form of the measure of test-takers' ability. The general expression of the item-response based ability measure is called General Total Score, which is available here: Research Total Score and General Total Score

If any question on this topic, please let me know. Thank you for your question.

Regards,

Nan Kong

What do the acronyms NIRT and MIRT stand for? What do you mean by "IRT is not consistent with CTT (classic test Theory)." Both theories define a latent trait variable clearly (and in the same way) based on stochastic measurement theory as the conditional expectation of an observed variable given the observational unit (person) variable. See Steyer (1989, pp. 50-51) as well as Steyer et al. (2015, Supplemental Text A: https://www.annualreviews.org/doi/suppl/10.1146/annurev-clinpsy-032813-153719/suppl_file/cp11_steyer_suptexta.pdf):

Steyer, R. (1989). Models of classical psychometric test theory as stochastic measurement models: Representation, uniqueness, meaningfulness, identifiability, and testability. Methodika, 3, 25–60.

Steyer, R., Mayer, A., Geiser, C., & Cole, D. A. (2015). A theory of states and traits—Revised. Annual Review of Cinical Psychology, 11, 71-98.

Article A Theory of States and Traits—Revised

(1) MIRT is Multi-dimensional IRT. NIRT should be MIRT (sorry for this typing error);

(2) In practice, IRT (2PL/3PL) and Total Score may not be consistent. In case that, for two examinees, say S1 and S2, there are

IRT ability (S1) > IRT ability (S2)

Total Score (S1) < Total Score (S2)

Which one we should trust for examinees' reported score?

Here, Total Score is the number of those correctly-responded items (testing questions).

In CTT, the number of the correctly-responded dichotomous items is a measure under the conditions:

(1) the items are jointly independent;

(2) all items have the same difficulty level;

(3) No partial credits.

General Total Score is defined to be a measure without above conditions.

i.e., with above three conditions, General Total Score is reduced to the number of the correctly-responded dichotomous items (with a constant coefficient). Therefore, General Total Score and the number of the correctly-responded dichotomous items is consistent.

I am not sure I understand. You show that different ways of measuring things can produce different orderings of what is being measured. Do you see this as a problem?

Hi, Daniel,

Mathematically, it can be proved that the mathematical expression of the ability measure (General Total Score) is unique except for a constant coefficient, i.e. if m is a measure, then c times m is also a measure, here c is a constant. Therefore, different measures will not "produce different orderings of what is being measured".

Total Score (the number of the correctly-responded dichotomous items) is a dichotomous item responses based measure (a special case of the General Total Score) while the laten variable of IRT is not a measure (the laten variable of IRT is an index that is positively correlated with Total Score), therefore, Total Score and the laten variable of IRT can produce different orderings of what is being measured, i.e. Total Score and the latent variable of IRT is not consistent.

In the case of 1PL model with dichotomous items, because Total Score is the sufficient statistics of the 1PL latent variable, the latent variable of 1PL and Total Score produce the same orderings of what is being measured. But, 1PL has the difficult parameter for each item while Total Score has not, "the latent variable of 1PL and Total Score produce the same orderings of what is being measured" implies that the 1PL ability "measure" is independent of the difficulty level associated with each item, i.e. orderings of what is being measured does not depends on the difficulty level of the items. Therefore, in scoring business, Total Score is still most popular way (only for dichotomous items) for high-stake scoring.

Thank you for your question.

Nan Kong