Dear John-Kåre Vederhus

attached please find the excel sheet for computing composite reliability.

Best,

Holger

You have to report composite reliability if you have performed CFA (Hair et al. 2009)

Hi John-Kare,

if reviewers want you to jump and clap your hands than you know what to do :)

Substantially, some information of the error of indicators is surely helpful but I would rather report the composite reliability or Omega. Alpha simply should never be used as it makes unrealistic assumptions and better methods are easy to implement.

Having low reliabilities, in addition, can indicate that - beyond a perhaps decent fit of the CFA - the model is invalid. In the extreme case, you could specify four completely unrelated indicators as reflective indicators of a single latent variable. As a result the model would fit (as there are no implications of the data which the model would speak against) but the loadings and errors would show that it does not work.

That having said, I personally don't think that the reliability is correctly located in a CFA/SEM context as the measurement error not only consists of random error but comprises all omitted further causes of the indicator (which from the perspective of the model would than be described or interpreted as "systematic error").

Composite reliability, which is computed based on the standardized loadings and error variance, would then not directly reflect reliability (=random error) but the portion of the variance attributable to the latent variable vs. "the rest" (i.e. random and other causes). It has IMHO more in common with validity than reliability.

As I became familiar and interested in methods by actually starting with SEM (before I knew more than the typical basics of regression and psychometric) I never did see so much value in solely focusing on random error (but rather "error" in general). Just knowing that a response is replicable, is not so interesting, in my point of view when you do not know that the response is all about.

But as Denny Borsboom once wrote "reliability is the poor man's validity". Perhaps, psychologists like it so much as you can easily calculate some fancy numbers :)

Best

Holger

Cronbach's alpha is necessary for you to report the reliablity of the instrument. It does not matter about whether you are conducting EFA or CFA.

Article Korean Patient-Perceived Satisfaction Scale of Community-Bas...

Article Psychometric Evaluation of a Patient Safety Competency Self-...

Dear Claire,

please allow me some slight, modest (partial) disagreement :)

I agree that inspecting measurement error is valuable but focusing on only random error (=reliability) makes no sense to me. I would like to know which part of the indicators variance is irrelevant and diverging from the information that I would like have. But this my personal view.

However, even if you are interested in reliability, alpha is the worst possible way to go, as the measures never meet the assumptions that alpha makes:

Cho, E., & Kim, S. (2014). Cronbach's coefficient alpha: Well known but poorly understood. Organizational Research Methods, 18(2), 207-230. doi:10.1177/1094428114555994

Huysamen, G. K. (2006). Coefficient alpha: Unnecessarily ambiguous; unduly ubiquitous. South-African Journal of Industrial Psychology, 32(4), 34-40.

Schmitt, N. (1996). Uses and abuses of coefficient alpha. Psychological Assessment, 8(4), 350-353.

Better would be to use Omega or composite reliability:

Deng, L., & Chan, W. (2017). Testing the difference between reliability coefficients alpha and omega. Educational and Psychological Measurement, 77(2), 185-203.

Best,

Holger

Thanks, all! Jeevan Jyoti , can you give me the title of the Hair reference?

I'm using Mplus and unfortunately it seems like it is not possible to get "composite reliability" as an output. Holger Steinmetz , can the CR be calculated manually relatively easy? Please explain as for a non-statistician.

Best

John-Kåre

Hi John-Kare,

the relevant paper is

Fornell, C., & Larcker, D. F. (1981). Evaluating structural equation models with unobservable variables and measurement error. Journal of Marketing Research, 18, 39-50.

The formula (equation 10) is absolutely easy and can be typed in Excel very fast. It is based on the standardized loadings and error variances that you find in the Mplus output.

I have an excel file on a private disc that I buried at home somewhere. If you have problems, I can send you the file. Tell me

Anyway the equation contains two quantities

a) the sum of the standardized loadings (which you then square)

b) the sum of the error variances

Together it is easy to see that this formula represents the classical test theory components true score variance (sum of loadings, squared) and error variance, and in particular the ratio of true score variance on the total variance.

Actually, I never understood, why Fornell & Larcker label this the "reliability of the construct" as reliability is per definition a characteristic of observed measures. I interpret this as the constant confusion of latent variables and composites by the marketing folks :)

HTH

Holger

I would be very grateful, Holger Steinmetz !! Here is one sub-factor with 3 indicators:

- sum of stx loading: 2.04 (0.54 + 0.77 + 0.73). Squared = 4.16

- sum of stx error variances (residual variances) = 0.71 + 0.41 + 0.47

I'm also confused about the terms used. Seems like the term latent factor sometime used even in a measure with formative indicators. Composite would be better.

jk

Hey John-Kåre Vederhus,

it depends on what you regard as formative measurement. The literature is not very precise here as the term comprises two different models/ways to operationalize theoretical concepts. Bollen & Bauldry (2011) distinguish between causal-formative measurement model and composite model. Both models looks quite similar but the latter assumes that indicators fully detemine the concept.

In the causal formative measurment model, the construct is a latent variable and the meaning of the construct is actually determined by its consquences although the name suggests something else (Aguirre-Urreta and Rönkkö, 2016). The consequences of the latent variable are required for identifiaction and the causal indicators are rather irrelevant for the measurement in particular in case of the MIMIC model. In this case, assessing construct reliability is useful as th construct scores are not fully reliable.

In contrast, in the composite model the theoretical concept is represented by a composite (linear combination of observable variables). In case of the composite model, I would not talk about measuring the theoretical concept. Moreover, in the composite model, assessment of construct reliability is most of the time meaningless as the construct scores are 100% reliable by assumption as the indicators are assumed to be free of measurement error.

I agree with Holger Steinmetz that Cronbach's alpha is not a good statistics to assess reliability due to its assumptions. However, it provides a lower bound for the reliability (Guttman, 1945; Sijtsma, 2009).

Best regards,

Florian

Bollen, K. A., & Bauldry, S. (2011). Three Cs in measurement models: Causal indicators, composite indicators, and covariates. Psychological methods, 16:3, 265.

Guttman, L. 1945. “A basis for analyzing test-retest reliability,” Psychometrika (10:4), pp. 255-282.

Miguel I. Aguirre-Urreta, Mikko Rönkkö & George M. Marakas (2016)

Omission of Causal Indicators: Consequences and Implications for Measurement, Measurement: Interdisciplinary Research and Perspectives, 14:3, 75-97, DOI: 10.1080/15366367.2016.1205935

Sijtsma, K. 2009. “On the use, the misuse, and the very limited usefulness of Cronbach’s alpha,” Psychometrika, 74:1, pp. 107-120.

interested

Dear John-Kåre Vederhus

attached please find the excel sheet for computing composite reliability.

Best,

Holger

Thanks, Holger Steinmetz !!!!!!!!!!!

Best

JK

CRONBACH himself should have a say in what and how his alpha should be used or whether it should still be used:

“I no longer regard the alpha formula as the most appropriate way to examine most data. Over the years, my associate and I developed the complex generalizability (G) theory (Cronbach et al. (1963); Cronbach et al. (1973); see also Brennan (2001); Shavelson and Webb (1991), which can be simplified to deal specifically with a simple two way matrix and produce coefficient alpha (Cronbach(2004), p. 403). Cited in N.M. Webb, R.J. Shavelson and E,H. Haertel (2006). “Reliability Coefficients and Generalization Theory.” Handbook of Statistics, Vol. 26, p. 2. ISSN 0169-7161.

LINKS:

Cronbach, L.J., Nageswari, R., & Gleser, G.C. (1963). Theory of generalizability: A liberation of reliability theory. The British Journal of Statistical Psychology, 16, 137-163. https://doi.org/10.1111/j.2044-8317.1963.tb00206.x

https://web.stanford.edu/dept/SUSE/SEAL/Reports_Papers/methods_papers/G%20Theory%20AERA.pdf

https://web.stanford.edu/dept/SUSE/SEAL/Reports_Papers/methods_papers/G%20Theory%20Hdbk%20of%20Statistics.pdf

In my knowledge Cronbach alpha measures for extracted factors is also important as it is a measure of reliability. The overall Cronbach used in EFA is an indicator of the scale used. It is important to mention cronbach alpha.

Dear Holger Steinmetz,

Can I use composite reliability for non-normal data or it is similar to coefficient alpha for these situations?

Best

Hi Matheus,

if you have corrected for non-normality (e.g., by applying the Satorra-Bentler or Yuan-Bentler correction) than I don't think there is any problems with it. Going one step further, it could even be that composite reliability could even be calculated with the non-corrected estimates as non-normality affects the model test and standard errors and not the estimates themselves.

Best,

Holger

In addition to Holger, it depends also on the degree of non-normality, so if you investigate categorical indicators, you can have a look at ordinal alpha suggested by Zumbo et al. (2007)

Best regards,

Florian

Zumbo, B. D., Gadermann, A. M., & Zeisser, C. (2007). Ordinal versions of coefficients alpha and theta for Likert rating scales. Journal of modern applied statistical methods, 6(1), 4

Thank you for your explanations

Follow

As far as I understand, Cronbach's alpha is not contradictory with confirmatory factor analysis (CFA). CFA tells you which items of the questionnaire can be grouped within the same factor in relation to their weights and if the factors are different between them or not. Cronbach's alpha tells you if the items of each factor are coherent within the factor. Thus CFA is more related to validity whereas alpha Cronbach is more related to reliability.

In order to be practical, although CFA has been carried out if the reviewers of your manuscript have suggested to include Cronbach alpha scores to confirm your scales are reliable enough, it is very easy to perform a reliability analysis, and thus you strength your document.