I am running a few reliability analyses on data regarding advertisements. The majority of the scales have items with corrected item-total correlations above 0.7. I was taught that items with item-total correlations above 0.7 are nearly measuring the entire scale itself. There is thus some redundancy.
Although the scales are reliable (90%+ Cronbach's Alpha), the quality of the scales can be improved. Do I throw out those highly correlated items -to -total or do I keep them?
Thank you.
Jolien
Dear Jolien,
As per researchers, the closer Cronbach’s alpha coefficient is to
1.0 the greater the internal consistency of the items in the scale. However very high value of alpha indicates that items used in the scale were comprehended to be almost same. As per George and Mallery (2003), the following rules of thumb is considered regarding reliability coefficient:
“Alpha > .9 – Excellent,
Alpha > .8 – Good,
Alpha > .7 – Acceptable,
Alpha > .6 – Questionable,
Alpha > .5 – Poor, and
Alpha< .5 – Unacceptable”
While increasing the value of alpha is partially dependent upon the number of items in the scale, it should be noted that this has diminishing returns. It should
also be noted that an alpha of .8 is probably a reasonable goal.
According to Nunnally and Bernstein (Nunnally J, Bernstein L. Psychometric theory. NewYork: McGraw-Hill Higher, INC; 1994. ) the acceptable values of
alpha, range from 0.70 to 0.95.
A high value of alpha is not considered a cause of cocnern but yes could definitely be considered to be worth relooking at the items that correlate very highly. A high value of alpha (> 0.90) may suggest redundancies and show that the
test length should be shortened. Thus, you can look out for items with correlation more than .90 to .95 and may try to reduce one or more items.
Best wishes
Dear Jolien,
As per researchers, the closer Cronbach’s alpha coefficient is to
1.0 the greater the internal consistency of the items in the scale. However very high value of alpha indicates that items used in the scale were comprehended to be almost same. As per George and Mallery (2003), the following rules of thumb is considered regarding reliability coefficient:
“Alpha > .9 – Excellent,
Alpha > .8 – Good,
Alpha > .7 – Acceptable,
Alpha > .6 – Questionable,
Alpha > .5 – Poor, and
Alpha< .5 – Unacceptable”
While increasing the value of alpha is partially dependent upon the number of items in the scale, it should be noted that this has diminishing returns. It should
also be noted that an alpha of .8 is probably a reasonable goal.
According to Nunnally and Bernstein (Nunnally J, Bernstein L. Psychometric theory. NewYork: McGraw-Hill Higher, INC; 1994. ) the acceptable values of
alpha, range from 0.70 to 0.95.
A high value of alpha is not considered a cause of concern but yes could definitely be considered to be worth re-looking at the items that correlate very highly. A high value of alpha (> 0.90) may suggest redundancies and show that the
test length should be shortened. Thus, you can look out for items with correlation more than .90 to .95 and may try to reduce one or more items.
Best wishes
Dear Jolien,
i would advise you to run a principal component analysis on each scale a remove items with a loading on the first factor of less than 0.4. The size of the Cronbach's alpha needs to be interpreted in context and with respect to the number of items in your scale.
Have a look at my article: https://www.researchgate.net/publication/280936182_Advice_on_Reliability_Analysis_with_Small_Samples?ev=prf_pub.
Technical Report Advice on Reliability Analysis with Small Samples
Hello Anantjyoti,
Thank you for your advice. You definitely helped me further.
Hello Peter,
Thank you for your comment.
I have run a few PCA's before doing any reliability analysis and all factor loadings are above 0.5. I did a couple of iterations.
I did a PCA, then a reliability analysis and looked at the item-total statistics table. Each time removing an item led to a higher cronbachs alpha,, I deleted that item and did a new PCA on the dataset minus that one item. The dataset contains 1690 respondents and 17 original variables. After running the PCA a couple of times I ended up with 2 factors with loadings all above 0.5. So I believe the loadings are sufficiently high.
- Badges
- Science topic
- Similar topics
- Mathematical Sciences
- Statistics
- Statistical Software
- SPSS