Dear Kenisha,

   Whether or not a measure works the same way across populations is an empirical matter. Alpha rises and falls with the intercorrelations between the items. Now items may not mean the same things to different people - so variations in the population are likely to show up this way. Even where items are equivalently interpreted, correlations are best evaluated when the range of values are at a representative maximum. So situations where samples are homogenous or otherwise have reduced variability should be expected to lower alpha as well. 

Dear Kenisha,

   Whether or not a measure works the same way across populations is an empirical matter. Alpha rises and falls with the intercorrelations between the items. Now items may not mean the same things to different people - so variations in the population are likely to show up this way. Even where items are equivalently interpreted, correlations are best evaluated when the range of values are at a representative maximum. So situations where samples are homogenous or otherwise have reduced variability should be expected to lower alpha as well. 

Have a look at the formula for the standardized Cronbach's alpha:

 Alpha=K*mean r/(1+(K-1)*mean r).

There are two (and only two) measures which influence α: K and mean r.

The first one K is the number of items (variables, subtests) selected to constitute your scale.

The second one is the mean of the non redundant correlations ((K-1)*K)/2 (the triangle of correlations above or below the diagonal in the correlation matrix).

A difference in Cronbach’s alpha is related either to the one or the other: alpha depends on the number of items and the average intercorrelation between items.

Dear Kenisha,

beside the already mentioned aspects you should take into account that the estiamte of alpha depend on sample characteristics of the sample. If you assess a very homogenous sample with a low overall variance of item scores, this will lower the correlations and therefore the estiamte of alpha. Therefore differences in the varainces of the assessed samples might lead to differences between the alpha estimates you observed. 

BEST

Cronbach's alpha is a measure for the sample. When a test manual reports an alpha it is for the sample used. Particularly with clinical measures often the sample used for the manual is very different than in a particular study. For example, using a general population sample versus a sample composed of people with a specific characteristic. The more homogeneous sample will often have lower standard deviations on the items, and this will tend to lower alpha.  Main thing to remember is alpha is not a characteristic of the test, it is a characteristic of the sample taking that test.

Thank you all for the great answers. Extremely helpful.

Best,

kenisha

Hi Kenisha,

Many times the alphas will vary between different groups. Statisticians have said that reliability can differ between samples for many reasons including those so far mentioned. I have never obtained the same alpha from different samples. It is as has been said that way people differ in their interpretations of an item and also because the  people are different

Reliability is a function of time, place and person so it is not surprising that the measure differs.  The more similar the alpha is between the studies, the more universal a measure.  It is the reason why measures need to be measured in different populations, places and times.  It is also the reason why you need to calculate it every time you administer an instrument.

Reliability and validity is not the property of the test, but of the scores. The scores may vary in different samples so may the reliability & validity.

Thank you all for responding to my question. I however have a follow up question. 

If my aim is to carry out a cross national comparison of an instrument and the alpha is extremely low (approximately .20) in one of the samples. Does this then mean that the instrument is unreliable in this population? or should I decide based on other information I have about the data?