Arjun - what is the hypothesis you intend to test? That Cronbach's alpha should be >0.70 is absolutely arbitrary. 

"with a 3-item scale"

be aware of the fact that cronbachs alpha not only depends on the correlation between the items in your scale but also on the number of items (= 3) in your scale.

with the same average correlation between the items cronbachs alpha would be higher if you have 4 items, 5 items, etc.

so i think 0.65 is quite ok for a 3-item scale.

Hi folks,

a) The .70 cut of is a myth, see

Lance, C. E., Butts, M. M., & Michels, L. C. (2006). The sources of four commonly reported cutoff criteria: What did they really say? Organizational Research Methods, 9(2), 202–220.

b) In most of the cases, a low alpha rather results from concepual heterogenety (the items measure different things) than low reliability

c) alpha is downward biased if the assumptions of tau-equivalence fail (i.e., the items measure the same thing and the loadings are equal) - this is almost always the case

However, altough Manfred is correct about the role of number of items, I would not concur with his conclusion :) If the assumptions hold (let's asume this), than a low reliability is a low reliability, may concern 10 items or 3. You (Manfred) are correct that this means that the 3 items themselves are more reliable than the 10 items but the conclusson for the whole scale is the same.

Best,

Holger

of course in scale construction the most important thing is to measure a construct in a valid way, to find content-related items which measure the construct in a broad sense.

if this job satisfaction scale is such a scale, it's ok.

but if it is a scale like: i'm happy with my job / i'm satisfied with my job / i'm not really satisfied with my job (R), then it's not a good scale and in this case cronbachs alpha is not the main problem...

Hi Manfred, here we *might* disagree again :)

you seem to have the goal to create composites consisting of different things ("...broad sense"). I, in contrast, have the goal to *measure* variables that reflect existing things. Hence, we might differ with regard to the ontological approach to constructs and measurement.

For my purposes/orientation, your proposed job satisfation scale would indeed be a good scale as each of these items is a valid measure of the same underlying phenomenon (i.e., the degree of job satisfaction) and hence, deviations of the responses to these items (1 - r) would indicate nothing more than measurement error.

Altough I also sometimes built composites, I always do this with some kind of regret because composite-creation is like inventing something what did not exist before. This contradicts strongy my view of science (..to investigate naturally existing phenomena).

In this regard, I am a huge fan of the work of John Cadogan (which is also on researchgate.

Lee, N., & Cadogan, J. W. (2013). Problems with formative and higher-order reflective variables. Journal of Business Research, 66(2), 242–247.

Best,

Holger

"For my purposes/orientation, your proposed job satisfation scale would indeed be a good scale as each of these items is a valid measure of the same underlying phenomenon"

to my opinion, this would be an "artificial" scale - one would not need such a scale but just ask one of these items. at least that's what i learned from the psychologists (e.g. markus bühner, who is now in munich).

of course some researchers develop such scales for pratical reasons :-) (they need it for structural equation modelling).

Things to consider:

You can still use the scale, but it is likely the correlations are attenuated over what they would have been if the scale had higher reliability.

Was this an existing scale, and if so, is the reliability typically better? If you translated it, or even used it in a country that is culturally dissimilar to where the scale was developed, it might have lower reliability.

Spector, P. E., Liu, C., and Sanchez, J. I. (2015). Methodological and substantive issues in conducting multinational and cross-cultural research. Annual Review of Organizational Psychology and Organizational Behavior, 2:9, 1.9-31. [Early view]

If items are written in opposite directions, this could result in reduced alpha because item correlations are affected, especially if translated to another language or used in a culturally dissimilar setting.

Spector, P. E., Van Katwyk, P. T., Brannick, M. T., & Chen, P. Y. (1997). When two factors don’t reflect two constructs: How item characteristics can produce artifactual factors. Journal of Management, 23, 659-678.

Hi Arjun.

As stated by others, you low alpha value could be a result of a low number of questions or poor interrelatedness between items (due to too much heterogeneity in the constructs). Many different researchers have different cut-off criterion thresholds, and quite honestly, you could spend weeks reading differing opinions. I agree that the value of .70 is somewhat arbitrary, but for most practical purposes, minimum values between .70-.80 are acceptable. You don't want values that are substantially lower than .70, which are indicative of an unreliable scale, high measurement error, as you may experience some real problems with your data. 

Also, alphas value should be interpreted within context of the research area. For example, minimum values of .80 are generally accepted in cognitive research, whereas in social research, values of .70 are acceptable. Given your research area of job satisfaction, I think that an alpha of .65 is ok considering your scale is only 3 items.

As others have said, the cutoff is totally arbitrary.  You certainly can't expect higher than that with only 3 items.  The number of items should be of far greater concern to you - you are unlikely to meet the needs of the assessment.

The issue of internal consistency was addressed - and I would say: resolved - by Bollen in 1984. His answer is that it depends on the relationship between the construct that is measured and the items that make up the scale. If the items measure things that are caused by that construct, high internal consistency is required. If the items measure things that make up the construct that is measured, there is no necessary relationship, and correlations between items of a scale might even be negative. This is also known as the distinction between an indicative and a formative scale.

Of course, all this doesn't resolve the arbitrariness of any cutoff, nor the issue that Cronbach's alpha is a lower bound (not an unbiased estimate) of reliability. (A better lower bound had already been published by Guttman when Cronbach published his alpha.)

Reference: Bollen, K.A. (1984). "Multiple indicators: Internal consistency or no necessary relationship?" Quality and Quantity, 18, 377-385.

I agree with Manfred and Holger.

Depends on your goals. For 3 items is good.

Have you tried the McDonald's omega?

http://www.personality-project.org/revelle/publications/revelle.zinbarg.08.pdf

http://www.personality-project.org/revelle/publications/zinbarg.revelle.pmet.05.pdf

Hey guys,

some comments:

1) With regard to the similarity of items: I agree insofar with Manfred that sometimes, items that are very similar in their wording can have problematic consequences: They may distort the factor model as items influence each other rather than an underlying construct influencing the items. Further, respondents may be very annoyed by responding to nearly the same items.

Hence, the crux of item generation is to invent items SIMILAR ENOUGH that they reflect the same underlying latent variable (not "construct") but NOT TOO similar to create those negative consequences. However, most scales that I investigated in the last 10 years contain items that are far more heterogenous and thus simply measure different things. No wonder, that 99% of the "established scales" showed bad results when tested with rigid methods (e.g., CFA).

2) With regard to the single - indicators (SI) vs. multi-indicator (MI) measures: The ONLY thingt that is relevant for enabling use of SI measures is the amount of measurement error that is a result of a) the clarity of the item formulation (--> random error) and b) conceptual closeness (--> systematic error)  - that is how directly the item focuses on the connotation of the focused latent variable. In this regard, SI measures of job satisfaction work well as the item "I am satisfied with my job" is good because it fulfills both criteria.

When researchers have the impression that SI measures can not reflect the "complexity of the construct" - than this is a sign of a poor construct conceptualization as THE concept has a single term and is ofteninterpreted as a singular phenomenon whereas the complexity (=multidimensionality) is treated on the measurement model. Multidimensionality belongs to the conceptual level.

3) Different cut offs for reliability: I would be interested why the reliability depends on the context. Measurement error (of independent variable) leads to endogeneity,  that is, bias of effects on dependent variables. Why should the severity of this bias depend on the context?

4) The fact that alpha raises with the number of items is no joke but makes totally sense: As long as the assumptions hold (tau-equivalence), adding more error-containing variables reduces the random error of the total sum. The problem is that the assumptions fail so fast. In my experience, it is impossible to create more than 3 items (=dissimilarity) that leads to these items measure the same latent variable.

Cheers

Holger

Hi All

Thank you all for your answers/comments. Let me clear my point more clear. This scale as Manfred point out has three items "All in all, I am satisfied with my job", "In general, O don't like my job" (R) and "In general, I like working here". From your discussions what I have inferred is that it is  better to have alpha value above .70 but this is not the absolute cut off value. When we have less number of items, alpha value less than .70 could be acceptable. 

Thanks once again.

Hi Arjun,

I would regard "all in all, I am satisfied with my job" as the FOCAL indicator, because exactly this aspect is the connotation of your intended latent variable.

For example these two studies show that this item is the best valid version and sufficient for measuring job satisfaction:

Nagy, M. S. (2002). Using a single-item approach to measure facet job satisfaction. Journal of Occupational and Organizational Psychology, 75, 77–86.

Wanous, J. P., Reichers, A. E., & Hudy, M. J. (1997). Overall job satisfaction: How good are single-item measures? Journal of Applied Psychology, 82(2), 247–252.

It seems that one of the other two items add something different leading to lack of correlation between these. Take a look at the correlation matrix: What's the outlier? I would suspect that the negatively worded item is the bad boy. I tend you avoid negatively worded items. There are tons of studies showing that such inverted  items measure different things than the same version of hte item formulated positively. See e.g.,

Green, D. P., and J. Citrin (1994). Measurement Error and the Structure of Attitudes: Are Positive and Negative Judgments Opposites? American Journal of Political Science, 38(1), 256-281.

Melnick, S.A. and Gable, R.K. (1990). The use of negative item stems: A cautionary note. Educational Research Quarterly, 14(3), 31–36.

Horan P. M., C. DiStefano, R. W. Motl (2003). Wording Effects in Self-Esteem Scales: Methodological Artifact or Response Style? Structural Equation Modeling, 10(3), 43-455.

Miller, T. R., and T. A. Cleary (1993). Direction of Wording Effects in Balanced Scales. Educational and Psychological Measurement, 53, 51-60.

Marsh H. W. (1996). Positive and Negative Global Self-Esteem: A Substantively Meaningful Distinction or Artifactors? Journal of Personality and Social Psychology, 70(4), 810-819.

HTH

Holger

So far you have received very good advice as they are consolidated in the literature.

However, one should not loose track of the fact that at the end what really matters is the researchers inteletual integreity. Thus, if you have obtained an alpha of .65 report this nunmber and complement it with other relevant data details as previous colleagues have suggested in their answers. 

it is yet to be demonstrated that humans are organized in a decimal manner. Mathematics (A cultural construction or humana invention) are only good as far as they help us systematicaly gather and report data. 

Provide rich methodological details in your publication so that others may be able to duplicate your study. This is the overall purpose of publishing. 

I agree that the Cronbach's apha coefficients tells whether or not the items under the various construct effectively measure the constructs. For appropriateness, different books recommend different values such reliability analysis. Pallant (2005) recommend that for a be

If the value of the Cronbach's alpha coefficient is greater than or equal to 0.70, the result is acceptable since the coefficient is equal to or exceeds the minimum threshold minimum threshold of 0.70 (Nunnaly 1978). But the threshold is arbitrary although it is widely accepted by the scientific community to obtain a satisfactory internal consistency.

Note that ccertains authors set a minimum threshold to 0.75 or 0.80, while others set it to 0.60. However, when the alpha value is 0.70, the standard error of measurement is equal to more than half (0.55) of the standard deviation of the distribution of the total score.

Dear Colleague,

Using a 0.70 or even lower represents poor measurement of any construct. I strongly recommend, have routinely apply in my NIH funded evaluation studies,  and have taught my graduate students that a study needs to set > 0.80 is a base Cronbach's Alpha. In Chapter 4, "Measurement and Analyses in Evaluation",  of my graduate level textbook "Evaluation of Health Promotion-Disease Prevention and Management Programs", 5th Edition, Oxford University Press, 2015 a detailed discussion and multiple examples of Scales with Factor and Psychometric Analyses confirming Validity and Reliability including Item Analyses, Internal Consistency Analyses, and Stability Analyses, are presented.

Meta-Analyses in the literature of Scales used by evaluation studies confirm that > 75% DID NOT produce evidence of the Validity and Reliability of the Scale used in the evaluation.

Hi Arjun

In general, I would say that .65 in three-item index or scale is sufficient enough, but unfortunately it would be a bit hard to substantiate in literature. Most of reliable sources, e.g. cited above by other colleagues, legitimate the .70 threshold. However, we have to remember that Cronbach's alpha is the measurement of (homo-)heterogeneity of the scale/index. In fact, it is all about the indicators (items) inter-correlations and their partial correlations to the whole scale/index you'd like to name after your construct. Nothing more, nothing less. As far as I know it doesn't matter what kind of construct you try to measure and what kind of indicators you'd use. It is just mathematics.

If you'd like to justify usage of an index which alpha is below .70 you can try to cite Field, A. P. (2009). Discovering statistics using SPSS (3rd ed.). Introducing statistical methods. London: Sage Publications, who analyse the issue of using Cronbach's alpha extensively. For example, Filed discussed there the problem of rising alpha along with adding items to index/scale; in other words - the more items, the highest alpha even if the 'real' reliability (I mean reliability based on a logic behind the set of indicators) is not rising.

I would also suggest you to check so called corrected item-total correlation (item-rest correlations) for each of your scale/index items. It should be over .50 which means that the item is at least moderately correlated to the whole scale/index.

The third problem is, in my opinion, if you asked literally these three questions: "All in all, I am satisfied with my job", "In general, I don't like my job" (R) and "In general, I like working here", and the Cronbach's alpha is below .70 you should double check the data, looking for typos and other errors (unnatural outliers, etc.). It seems highly unlikely that answers to such three nearly identical questions were not nearly perfectly correlated... The Cronbach's alpha should be over .85 here or even over .90; as each of these three seem to be focal indicators (as Holger Steinmetz  said above). I would bet that the problem occurred somewhere in the reversed item, either in data input (reversed was not reversed properly) or the participants had a problem to answer in the double negation (I don't like... and then the likert type item "not agree"...).

Hi Michal

While going back to data entry, I found some errors. After correcting errors and removing outliers, the value of Cronbach's alpha increased to .77. However, when I subjected this data to CFA using LISREL, I didn't get the values of Fit Indices, rather the output showed "The Model is Saturated, the Fit is Perfect". Wondering how to interpret it. Is something wrong with my analysis.

I have attached the output herewith. 

Hi Arjun,

you should read a bit about identification in CFA/SEM models. Briefly, models are identified, if at least as many observed elements (i.e., covariances and variances of the observed variables) as estimated parameters exist. If there are more observed elements than parameters, the model is overidentified and can be tested. Your model is "just identified" - that means you have six observed elements (3 variances and 3 covariances) and also 6 parameters (3 error variances, 3 loadings; the latent variance is fixed to one).

Hence, you cannot test the model, as the df are 0. Enlarge the model by incorporate other observed/latent variables. Then you get a fit. As you already moved into the CFA world, you can easily calculate the composite reliablitity which is better than alpha as it does not require equal loadings, see

Fornell, C., & Larcker, D. F. (1981). Evaluating structural equation models with unobservable variables and measurement error. Journal of Marketing Research, 18, 39–50.

Best,

Holger

Arjun:  Let me try to answer your question.  The following formula specifies the relationship between validity and reliability.

     Rxy

The stronger the Cronbach's Alpha Coefficient the better the inter correlation between items and constructs. However, different books have different cut-offs points for it which should be considered. 

Hi David

I'm not clear about it. Would you please suggest me books or articles. 

     I agree with those who stated that it is arbitrary to demand that Cronbach’s alpha value exceed .70. Acceptable values would depend on the nature of a study (e.g., in an exploratory study it is ok if the Cronbach’s alpha is lower than .70 – see Hair et al., 2006) and also academic discipline (e.g., I have noticed that in education and business studies it is preferred if the value above .70). As some researchers here have commented, the length of scale would also affect the value, and a 3-item scale with Cronbach’s alpha of .70 is quite ok for me (if I were reviewing a paper).

     As to some literature on this topic, which you have requested to suggest, Nunally (1967) maintained that in theoretical studies, even modest reliabilities of .60 or .50 may be acceptable. To agree, Hair et al. (2006) proposed that though “generally agreed” lower limit for Cronbach’s alpha value is .70, it may decrease to .60 and still be acceptable, especially in exploratory studies and in research in the Social Sciences. Furthermore, Aron and Aron (1999) proposed that in research in psychology, Cronbach’s α of .60 or even lower could be adequate; however, values exceeding .7 are preferable (Aron & Aron, 1999).     

References

Aron, A., & Aron, E. (1999). Statistics for psychology (2 ed.). Upper Saddle River, NJ: Prentice Hall. 

Hair, J. F., Black, W. C., Babin, B. J., Anderson, R. E., & Tatham, R. L. (2006). Multivariate data analysis. Prentice Hall Pearson Education.

Nunnally, J. C. (1967). Psychometric theory. New York: McGraw-Hill.

Best luck with your research!

I believe that a Cronbach Alpha < 0.70 is an inadequate measure of any "construct" (#1). Calling a project, an Exploratory Study, does not eliminate or reduce the salience of measurement principles and standards. Multiple published Meta-Analyses (MA) of the "Validity and Reliability of Psych-Social Scales" consistently confirm > 75% DO NOT meet acceptable standards of measurement.  A critical  issue, defined  by the MA's, and raised in this discussion, is the typical lack of representativeness and the inadequacy of the samples (small sample size/no power) presented by most social and behavioral studies (SBS). It is an illusion that a "3 question scale" based on the responses of a small (

It is a fact that psychology and behavioural science research mostly (at least as for now) relies on small populations, which is well accepted and well justified  (Kaplan, 2000; Kassin, Fein, & Markus, 2011; Taylor, Peplau, & Sears, 2003; Whitley & Kite, 2013). Besides, there are laboratory experiments that simply cannot include a statistically representative population. 

Each method and each statistical procedure has its owns caveats and weaknesses. There are also wider epistemological issues. To claim that one has achieved a "true measure" with having a Croncha's alpha = .60 is a folly. As it would be with a Cronbach's alpha of .89.

The basic point to be addressed in this discussion is the meaning of the Cronbach Alpha score, and that has to do with reliability, which, in turn, is really a measure of how precisely are you measuring a construct. In a sense, one can interpret it as if it were an estimate of the noise-to-signal" ratio. A value of zero means that there is only "noise", whereas a value of one would indicate that a perfect assessment of the "signal" is being made.

That being the case, the question becomes one of how much "noise" one is willing to put up with, something which, of course, is quite subjective. In hypothesis testing there is a similar issue regarding the probability of the null hypothesis, which was solved by means of a consensus over an arbitrary value of p

I think 0.7 or more is often recommended as the best index for reliability abeit some studies use 0.6 or more. 

A Cronbach Alpha value of < 0.70 confirms  < 50% error. How to produce a reliability coefficients of > 0.80, Internal Consistency and Test-Retest, hav been established for > 50 years. I agree with Dr Srinivas about Item r values.

I strongly believe that the "Nunnally" recommendation for  r  > 0.70 should be increased to > 0.80. Why would you want to produce a set of question (Not a Scale ???) with an r < 0.80.  Ask: Do you really think you are measuring any construct with 3-4 questions with such a high level of error?Where is the discussion about "Validity"

As an US-NIH Principal Investigator and Professor in Academic Health Sciences Centers for 40+ years, This is one of many continuing reasons why the "Social Sciences (SS)" are held in such low esteem by the biological-basic sciences.

Some authors have considered alpha values of .6 or above acceptable for a multi-item measure to be internally consistent.

There is nothing mandatory about a reliability of .7 to test hypotheses.  The real issue in hypothesis testing of a relationship is validity.  Reliability of your measurement instruments, whether IV or DV, creates an upper bound on validity, however.  The relationship between validity and reliability can be expressed as follows:

                rxy

It would seem that the basic point is what you want to achieve. For the goal of having a psychometric instrument that is precise and trustworthy enough for clinical use (or something analogous to that in a non-psychological context), it would probably be best to go for higher values of Cronbach Alpha (say, .70 or .80+). However, if the aim is to measure a trait with enough accuracy to establish the existence of a relationship with other traits, for research purposes (e.g., to test a model or to find results to guide the drafting of one), then you could very well accept lower values (something like .60 or even less) IF an acceptable  correlation occurs. On the other hand, if the Alpha score is low AND you have no significant correlation with other variables of interest, then you got no way of determining whether the absence of the correlation was due to the lack of an association or simply due to error of measurement.

Bottom line is that, as always, the saying from George Edward Pelham Box applies:

Remember that all models are wrong; the practical question is how wrong do they have to be to not be useful.

One might also remember another important quote from Box:

Since all models are wrong the scientist cannot obtain a "correct" one by excessive elaboration. On the contrary following William of Occam he should seek an economical description of natural phenomena. Just as the ability to devise simple but evocative models is the signature of the great scientist so overelaboration and overparameterization is often the mark of mediocrity.

Basically, you go with what can be shown to work.

Adding to what I said before, it is also important to distinguish between agreeing with an author such as Nunnally, Hair et al, or whomever that might be and blindly accepting said author's statements based only on his/her personal authority. One has to have in mind what is being done mathematically and logically, plus the scientific implications.

A completely separate matter is the adherence to a strong consensus when there is one, such as is the case of p

Also, remember that a very high Alfa might indicate that the variables are measuring exactly the same question. If there's a correlation between different variables (i.e. significantly different in their content and formulation) I don't see a problem with an Alfa bellow .7.

Yes, and please refer to:

1. Landau, S. and Everitt, B. S.(2004). A Handbook of Statistical analyses using SPSS.

2.Howitt, D. and Cramer, D.(2008). Introduction to SPSS.

Regards,

Zuhair

"There is no sacred level of acceptable or unacceptable level of alpha. In some cases, measures with (by conventional standards) low levels of alpha may still be quite useful" Ref: Use and Abuses of Coefficient Alpha, Schmitt 1996

Would anyone have a source to cite regarding the acceptability of a >.6 alpha?

Hi Laura,

unfortunately, even the .70-"threshold" seems to be a myth, see

Lance, C. E., Butts, M. M., & Michels, L. C. (2006). The sources of four commonly reported cutoff criteria: What did they really say? Organizational Research Methods, 9(2), 202-220. doi:10.1177/1094428105284919

Having said that most examples of low alpha simply result from divergences of its basic assumption (essential tau-equivalence which is - in factor parlance a) the items reflect the same latent variable and b) their loadings are equal.

If "a" holds but not "b", computing the composite reliability may be a better solution and gives a more realistic and often substantially higher alpha. Most cases of low alpha, however, are a result of not even "a" being true - that is the scale is a mess of different things lumped together.

Hence, check the factor structure of the scale.

Best,

Holger

I suspect that some notions about Cronbach's alphas fall into the category of myths. One of these is that the higher an alpha value, the better. A high alpha value can be quite meaningless if it is based on a large number of items, say, more than 20, yet researchers frequently cite high alpha values as if those values are somehow great achievements or reassurances (reassurances of who knows what).

It would appear that many researchers cite Cronbach's alpha values simply because many other researchers do so. In my view, this is a practice, based primarily on momentum, that urgently requires re-examination, particularly because many alpha values are likely to be meaningless.

If alpha values are very low, depending on the number of items in scales, they could indicate an undesirable lack of association among individual items - so they can serve a useful purpose.

I notice that a reference to the following article has already been provided above. Here is the full reference.

Schmitt, N. (1996). Uses and abuses of coefficient alpha. Psychological Assessment, 8, 350–353. http://dx.doi.org/10.1037/1040-3590.8.4.350

I strongly recommend it.

t would appear that many researchers cite Cronbach's alpha values simply because many other researchers do so. In my view, this is a practice, based primarily on momentum, that urgently requires re-examination, particularly because many alpha values are likely to be meaningless.

If alpha values are very low, depending on the number of items in scales, they could indicate an undesirable lack of association among individual items - so they can serve a useful purpose.

Sijtsma, K. (2009). On the use, the misuse, and the very limited usefulness of Cronbach’s alpha. Psychometrika, 74, 107–120. ����

The following references might also be helpful:

Cortina, J. M. (1993). What is coefficient alpha? An examination of theory and applications. Journal of Applied Psychology, 78, 98–104. http://dx.doi.org/10.1037/0021-9010.78.1.98

Sijtsma, K. (2009). On the use, the misuse, and the very limited usefulness of Cronbach’s alpha. Psychometrika, 74, 107–120.

Both authors provide revealing insights about the nature of alpha as well as helpful background information.

Very beneficial responses. Thanks all.

If your alpha is too low (