I've been working with IRT and usually I interpret the item information function and test information function based on the purpose of the measure I'm developing. A screening measure will most likely have more information about the low levels of ability. A performance based test might have more of a balanced TIF or might need a curve located to the high levels of trait.
One thing that picked my attention is that we find people saying on papers that item "y" has little information at any given location or that the test provides little information about the trait continuum, etc. So... my question is are there guidelines for the amount of information that an item or a test should provide? Are there any rule of thumb to interpret how much information should the TIF peak at?
I understand that the peak of the TIF or IIF isn't the most important information as we need to pay attention to the area to understand the distribution of the information, still I had that question on my mind....
Elaborating on Ivanouw's answer, for dichotomous models like the 3PL, the standard error of measurement (SEM) is equal to 1/sqrt(TIF(theta)). So, if your TIF has a height of 10, then SEM = 1/sqrt(10) = 0.316. So if that were at the point theta=1, that means that (assuming normally distributed errors) the 95% confidence interval for the true theta with an observed theta = 1 is [0.38,1.62].
So, that's your rule-of-thumb... You pick how narrow you want the 95% CI and work back to the height of the TIF required. I think SEM
Hi Pedro
As you mention, the question depends on several considerations, among which besides the peak are also the form of the TIF function. However, using the excellent book 'Item response theory for psychologists' by Embretson & Reise as source (p. 270) you could use the following as a guide:
A test information of 10 equals a SE around .31, which corresponds to a reliability coefficien of 0.90
Best
Jan Ivanouw
Good discussion Pedro.
Another good sources is Embretson & Reise. They discuss the spread of b values as well as the peak of a. It can be useful to sue a rough minimum IIF score (and flat IIC) to eliminate items that don't provide much information, but if lower IIF items are the best ones for capturing very low or very high theta levels, than you have to make important judgement calls. The sample size and characteristics can also help with your confidence in the IRT results.
Best wishes with your work!
Embretson SE, Reise SP. (2000) Item response theory for psychologists. Mahwah, NJ: Lawrence Erlbaum Assoc.
Thank you all! I also found out that ltm actually calculates the area bellow the TIF which is quite useful. I've been finding a lot of use for TIF to investigate some theoretical problems related to measurement.
Elaborating on Ivanouw's answer, for dichotomous models like the 3PL, the standard error of measurement (SEM) is equal to 1/sqrt(TIF(theta)). So, if your TIF has a height of 10, then SEM = 1/sqrt(10) = 0.316. So if that were at the point theta=1, that means that (assuming normally distributed errors) the 95% confidence interval for the true theta with an observed theta = 1 is [0.38,1.62].
So, that's your rule-of-thumb... You pick how narrow you want the 95% CI and work back to the height of the TIF required. I think SEM
Alan, your answer was incredible. Sometimes we do look for rules of thumb while we lose sight of the main problem. Thank you for remembering me about that.
- Badges
- Science topic
- Similar topics
- Methodology
- Research Methodology