Point biserial correlation is used to to determine the discrimination index of items in a test. It correlates the dichotomous response on a specific item with the total score in a test. According to the literature items with Point biserial correlation above 0.2 are accepted. According to Crocker (Introduction to classical and modern test theory, p.234) the threshold for point biserial correlation is 2 standard errors above 0.00 , and the standard error could be determined by (1/sqrt(N)) where N is the sample size. What is not clear to me is that in tests we need items that have high discrimination (Correlation) and if Point biserial correlation is a special case of pearson correlation that means that by accepting 0.2 as a threshold we are accepting the fact that the coefficient of determination is 0.04 and total score is only capturing 4% of item variance.
Point-biserial correlations of items to scale/test totals are a specific instance of the broader concept of the item-total correlation (ITC).
The size of an ITC is relative to the content of the measure. If you are using a personality or attitude instrument, you are likely to have much higher correlations than 0.20. However, for ability tests with multiple-choice responses (i.e., guessing) you are likely to have lower correlations, perhaps many below 0.20.
I have never heard of a specific threshold (0.20) and I don't think it makes sense. Items with ITC's of 0.199 and 0.201 are essentially the same quality. I also think that such a threshold over-emphasizes ITC which is only a single indication of quality. For example, if such a threshold lead to a test without any easy, or hard, items, that would be a mistake.
Also, I think whatever authority quoted 0.20 was saying that you shouldn't use items with ITC's below 0.20, not that an ITC of 0.20 was a good ITC.
Should we be worried that the ITC "explains" only 4% of the item variability? No, not at all because we should NEVER make decisions based on individual items. Rather, we should always make decisions based on SCORES which have suitable reliability for a sample/purpose. But you can draw this connection: If individual items have ITC's around 0.20 or 0.30, you are going to need quite a lot of them to reach an acceptable level of reliability. If your ITC's were much higher, you could get away with much shorter tests. Indeed, most ability tests are 30 or more items, while most attitude and personality scales have much shorter lengths (Hinchin had a paper around 1995 where he found the modal scale length to be 5 for organizational measures--probably mostly attitudes).
Assuming r-pbis = .20 is derived from 2 standard error, indicating that the correlation is statistically significant (p
The mathematical procedure of rbis (= pearson correlation between an individual 0/1 item and the total score) makes it happen that, with a perfectly/deterministically discriminating (theoretical) item [defined by the fact that the score is perfectly discriminating and only the best test-takers give the correct answer = Guttman type of item] of very easy or very demanding type (95 - 99% correct or 1-5% correct), the discriminating power LOOKS low because of the low correlation. With 100 cases with perfectly discriminating score (score from 0 to 99 and different with each case) and only one correct case (1% of correct answer By the best test-taker) rbis = 0.17. In the practical classical item analysis we can bear the rbis = .20 with ultimately difficult and easy items because of knowing the calculative challenge in the estimation. BUT, when the item is of mediocre difficulty level, we would expect higher (rbis = .5 would be very nice). Low (round rbis = .20) with a medium difficult item means just that the item is poor.
With a short test (couple of items) we need much higher rbis than 0.20 to raise the reliability even at the level of alpha = .60. In a long test, there can be some items of this low discrimination and, still, the reliability may remain sufficient [but in that case you can add ANYTHING in the score - like postal codes and still the reliability would be high]. If willing to raise the reliability higher than .80 with relative short test, the correlations need to be at the level rbis = .50, I guess. This is easy to test.
Point-biserial correlations of items to scale/test totals are a specific instance of the broader concept of the item-total correlation (ITC).
The size of an ITC is relative to the content of the measure. If you are using a personality or attitude instrument, you are likely to have much higher correlations than 0.20. However, for ability tests with multiple-choice responses (i.e., guessing) you are likely to have lower correlations, perhaps many below 0.20.
I have never heard of a specific threshold (0.20) and I don't think it makes sense. Items with ITC's of 0.199 and 0.201 are essentially the same quality. I also think that such a threshold over-emphasizes ITC which is only a single indication of quality. For example, if such a threshold lead to a test without any easy, or hard, items, that would be a mistake.
Also, I think whatever authority quoted 0.20 was saying that you shouldn't use items with ITC's below 0.20, not that an ITC of 0.20 was a good ITC.
Should we be worried that the ITC "explains" only 4% of the item variability? No, not at all because we should NEVER make decisions based on individual items. Rather, we should always make decisions based on SCORES which have suitable reliability for a sample/purpose. But you can draw this connection: If individual items have ITC's around 0.20 or 0.30, you are going to need quite a lot of them to reach an acceptable level of reliability. If your ITC's were much higher, you could get away with much shorter tests. Indeed, most ability tests are 30 or more items, while most attitude and personality scales have much shorter lengths (Hinchin had a paper around 1995 where he found the modal scale length to be 5 for organizational measures--probably mostly attitudes).
I guess you have to remember back in the old days before desk top computer processing became available and calculations were done by hand or by punch-cards, you would want a rule of thumb that was easy to derive. So rpb=.20 is just a convention for ensuring that the item has a non-chance positive association with the total. Of course arbitrary cut-offs are just that--arbitrary. If you rounded the example of .199 and .201 to .20 you would make the same rule of thumb decision. However, with modern computing you can determine rpb very quickly and accurately. Indeed, if the correlation is >0.00 you could probably keep the item even though it might not help discriminate candidates and it might never get used in any kind of selection or adaptive testing. Rules of thumb are how our industry started over 100 years ago when statistical and computing technology was primitive....keep it in mind when approaching conventions.
Yeah, this was just and easy, semi-arbitrary rule of thumb. Some types of assessments will have higher point-biserials than other types of assessments, and you can be flexible with what you consider to be minimally acceptable. The range of ability in the pilot sample affects Rpbis, as does difficulty of the item (substantial floor and ceiling effects).
Dear all,
I think that there is also a mathematical relation between rbis and the number of items in a test.
Suppose I would take a test and split it in two parts. the following happens: In the split test, rbis of a particular item would be higher as the influence of that particular item on the total score of the split test would be higher. Am I right?
For that reason, in the Netherlands often the rbis is computed over the total score minus the score of the specific item to exclude the effect of the item itself.
Spearman-Brown prediction formule can be somehow incorporated I guess for the relationship between test length and value of rbis.
Maybe someone has some more ideas?
- Badges
- Science topic
- Similar topics
- Medicine
- Physiology
- Biosignals
- Biomedical Signal Processing
- Statistical Analysis