Javier, to add to Robert's and Holger's contributions, alpha is generally used because it is implemented in statistical programs (e.g., SPSS) and it is recommended in several textbooks. Researchers generally do not know that alpha is based on assumptions (unidimensionality, tau-equivalence). These assumptions are seldom fulfilled in empirical research, but researchers often do not care or they do not know this.

For multidimensional constructs, using alpha does not make any sense - a multidimensional construct is of course not unidimensional.

I think the problem is that people do not realize that there assumptions that need to be fulfilled, and even if they do, they are not tested. If the investigated construct is multidimensional and reliability of subscales and the total scale should be estimated, omega coefficients are not easily available, that's why alpha is used. Furthermore, reliability coefficients of multidimensional scales are just gaining popularity, but at the moment, they are seldom taught at the universities.

Kind regards, Karin

Javier García Reynaud, your question is a good one and you've raised an important issue - and perennial problem.

In a nutshell, I think that obtaining an overall coefficient alpha that spans several different domains is essentially nonsense. It is a habit that I believe unthinking researchers engage in because many other (unthinking) researchers have done exactly the same beforehand.

One indication of the silliness of obtaining an overall alpha that spans different domains is that the overall alpha is almost always higher than are the separate alphas. How can that be logical? Surely, if the overall alpha relates to disparate domains it should be LOWER than the alphas for each domain. It's because alpha is highly dependent on the number of items being analysed (the more items, the higher the alpha) - something that many researchers don't seem to appreciate.

Incidentally, the metric you refer to should not really be called Cronbach's alpha. Even Lee Cronbach himself indicated that. It is more appropriately called coefficient alpha, or just alpha for short.

I suspect that many researchers don't know enough about alpha and its limitations. They just cite alpha values, like some kind of mantra, because everyone else does that - without knowing why they do it.

If you'd like references regarding the above, please feel free to ask me!

Hi Robert,

after you revealed their trick (add items until alpha is nice), fear for your life :)))

There was a nice quote by Denny Borsboom that "reliability is the poor man’s

validity coefficient" (p. 31, which seems to be quote form Rozeboom).

Validity involves something hidden, uncertain, hard to evaluate. Instead, you can estimate alpha easily. Psychologists hate the first, and love the second. That's how rituals are born.

After all, science is often a bit like a cult (the "cargo cult" by Feynman) and the followers adhere to the rules of the high priests :)

The only hope I have and that keeps me fighting against windmills is that sometimes things change when enough people to the right thing.

Best

Holger

Borsboom, D. (2005). Measuring the mind: Conceptual issues in contemporary psychometrics. Cambridge: University Press.

Holger Steinmetz, thanks for your support. I find it really tiresome that so many researchers rattle off coefficient alpha values as if those values always carry some important meaning. I suspect a large motivation for that, as you suggest, is that coefficient alpha is easy to obtain.

In a recent publication, I insisted that we not calculate a coefficient alpha because, with 24 items, an alpha would be meaningless.

I'll take this opportunity to provide some references about alpha that I have found to be informed and informative:

Cho, E. (2016). Making reliability reliable: A systematic approach to reliability coefficients. Organizational Research Methods, 19(4), 651–682. https://doi.org/10.1177/1094428116656239

Cho, E., & Kim, S. (2015). Cronbach’s coefficient alpha: Well known but poorly understood. Organizational Research Methods, 18(2), 207–230. https://doi.org/10.1177/ 1094428114555994

Schmitt, N. (1996). Uses and abuses of coefficient alpha. Psychological Assessment, 8(4), 350–353. https://doi.org/10.1037//1040-3590.8.4.350

Taber, K. S. (2018). The use of Cronbach’s alpha when developing and reporting research instruments in science education. Research in Science Education, 48(6), 1273–1296. https://doi.org/10.1007/s11165-016-9602-2

Javier, to add to Robert's and Holger's contributions, alpha is generally used because it is implemented in statistical programs (e.g., SPSS) and it is recommended in several textbooks. Researchers generally do not know that alpha is based on assumptions (unidimensionality, tau-equivalence). These assumptions are seldom fulfilled in empirical research, but researchers often do not care or they do not know this.

For multidimensional constructs, using alpha does not make any sense - a multidimensional construct is of course not unidimensional.

I think the problem is that people do not realize that there assumptions that need to be fulfilled, and even if they do, they are not tested. If the investigated construct is multidimensional and reliability of subscales and the total scale should be estimated, omega coefficients are not easily available, that's why alpha is used. Furthermore, reliability coefficients of multidimensional scales are just gaining popularity, but at the moment, they are seldom taught at the universities.

Kind regards, Karin

Javier

There are brilliant contributions here!

Why do people calculate alpha for a multi-dimensional instrument? A cynic may think that this is done because the answer tends to look good! Unmodeled sources of systematic variation will tend to inflate alpha for a set of indicators. We showed this in a set of simulations..

Article Coefficient alpha: A useful indicator of reliability?

and it was also examined in a much more elegant fashion in...

Article A note on using alpha and stratified alpha to estimate the r...

Many years ago Raykov made a convincing case for estimating reliability as part of a factor analytic study - I suppose this as gained some traction as it is now not uncommon to see composite reliability being reported, but not as frequently as alpha!

Article Estimation of Composite Reliability for Congeneric Measures

I suppose it takes time for people to change habits, and this is not helped if reviewers for journals either don't understand or don't care about such issues. Indeed, it would appear to be an uphill struggle as it also seems to be ingrained to report reliability and validity as properties of actual scales or questionnaires rather the scores. Not quite the medial equivalent of leeches and humours, but (speaking as a psychologist) we don't often demonstrate sound practice or knowledge on issues of measurement.

Mark

When a questionnaire is multi-dimensional, the alpha will go down. It’s the best to identify dimensionality first and compute an alpha separately for each to boost the alpha.