The total score, which is the number of the correctly responded items (questions),is the major tool for student's reported score (high stake scoring). How to create the high stake scores for each student for the general case that (1) the items (questions) have the different difficulty levels and (2) the items have partial credits and (3) items are interactive among their any combination???
to score the items which have different difficulty levels you must classify these items into groups , every group contains the elements which have the same properties, we can in this classification produce results show the percentage of every group, so we can also determine the limits of the success or failure of the case we are studying.
IRT (Item Response Theory) can not correctly score those items (questions) of different difficulty levels and the interactions among the items because:
(1) In 1PL (Rasch model), the total score (the number of those correctly responded items ) is the sufficient statistics of the theta estimates (ability estimates as called in IRT), therefore, the theta estimate actually is a (non-linearly) transformation of the total score -- difficulty level associated with each item does not play any role in theta estimates:
(2) In 2PL/3PL, the summation of the discrimination estimates is the sufficient statistics of the theta estimates, therefore, the theta estimate is fully dependent on the summation of the discrimination estimates -- difficulty level associated with each item does not play any role in theta estimates. Even worse, so many paradoxical results along with 2PL/3PL.
(3) In practice, IRT can ONLY be used for low-stake scoring. IT IS WRONG FOR HIGH STAKE SCORING WITH IRT/MIRT. Please online search the key word "paradoxical results with MIRT" for more detail.
(4) IRT/MIRT can not correctly handle those interactions among any combinations of the items, otherwise IRT/MIRT should have their complete decomposition theory. One direct consequence of "no decomposition theory in IRT" is that, in MIRT, only those scores associated with sub-scales are estimated, but the total score associated with all items is not available.
Nan, I am a little confused by some of your statements. Maybe it is just me. IRT just refers to a latent variable model with binary (sometimes ordered) manifest variables and metric latent variables (e.g., Bartholomew and colleagues latent variable modelling books). It is often called latent trait modeling (or modelling by Bartholomew) , which provides a more useful indication of how it relates with other procedures. Are you saying that you cannot include interactions among different manifest and latent variables of latent variable models?
Daniel,
Thank you for your question on this.
The "interaction" in the question is those interactions among any combination of the ordered manifest variables. Those manifest variables are also called "items" in business. Those items may or may not be binary -- they could be dichotomous items (binary manifest variable ) or polytomous items (the manifest variables with partial credits, for example, the item with possible ordered values 1,2,3,4,5 ...). The relation between the item/manifest variable and latent metric variable is fully described with the parametric structure of the model you use. For example, in (1PL) IRT, the summation of the binary manifest variables (with values: 0 or 1) is the sufficient statistics of the latent matric variable (the theta parameter in IRT) which implies that the latent variable estimate is an non-linear function of the summation of the binary manifest variables (called Total Score). In IRT, the interactions among the manifest variables or items are not included, Otherwise, IRT should has its complete decomposition theory, but, as we know, IRT has no its decomposition theory. One consequence of "no decomposition theory in IRT" is that, in Multidimensional IRT, only those matric latent variables associated with sub-scales are estimated (sub-scores), but the latent metric variable associated with all items (total score) is never available. Another consequence of "no decomposition theory in IRT" is that no way to measure the shared ability or unique ability which are highly expected in business.
The solution to this question is called theory of ability measure (also called general total score) which can be found under this Researchgate account. Thank you again for your interest in this topic.
- Badges
- Science topic
- Similar topics
- Psychology
- Psychometrics