Hello Matthias,

I'm unclear as to what it is you think the problem might be; your query is pretty sparse with respect to detailed information concerning your model and the results. Barring nonsensical values (e.g., "1.78" or "300%"), how can estimated reliability be "too high?"

If you're asking, are two constructs in your model really just the same thing, you can easily evaluate that by comparing a second model which combines them into a single factor (or, for "nearly the same," positing a second order factor that subsumes both).

As well, is there some compelling reason for opting to use partial least squares sem vs. covariance-based sem?

Can results change as your sample size changes? Yes, of course they can. Whether they will is another question altogether.

Good luck with your work.

Hi David Morse,

Thanks for your answer.

As suggested in Hair (2017) values above 0.90 and certainly above 0.95 are not desirable for composite reliability as they indicate that all items measure the same phenomenon and thus do not provide a valid measurement of the construct.

In my model 2 constructs clearly exceed the limit for the composite reliability (and cronbachs alpha). (See screenshot attached)

My question is rather I should change the items measuring the constructs? The existing items I picked from literature and adopted them for medical video consultation context. Other items might perform better in this regard?

But then I would have to "trash" my existing data and start from 0, right?

I'm basically building up on the model from Schmitz et al. (2020).

Thanks for your help!

Hair, Hult, G. T. M., Ringle, C., Sarstedt, M., Richter, N., & Hauff, S. (2017). Partial Least Squares Strukturgleichungsmodellierung (PLS-SEM): Eine anwendungsorientierte Einführung.

Schmitz, A., Díaz-Martín, A. M., & Yagüe Guillén, M. J. (2022). Modifying UTAUT2 for a cross-country comparison of telemedicine adoption. Computers in Human Behavior,130, 107183. https://doi.org/https://doi.org/10.1016/j.chb.2022.107183

It does not make sense to say reliability is too high. This is complete nonsense. In an ideal world, measurements would have perfect reliability (that is, reliability = 1.0, no measurement error).

A different (and important) question is whether a reliability estimate of .95 is realistic. If it doesn't seem plausible then it might indicate a problem with the statistical model (misfit, misspecification) or with the data. But in principle, there is absolutely nothing wrong with a reliability of .95. On the contrary, such a high reliability is highly desirable as it indicates high precision of measurement. And there are certainly scales out there that have such a high reliability in certain populations.

Hello again Matthias,

If your indicator/manifest variables have a well-vetted rationale for inclusion in the first place, I believe that worrying over "too high" values for estimates of composite reliability is a waste of time and energy.

First, Hair et al. note that composite reliability (CR) estimates are upward biased (whereas, according to Lord and Novick, coefficient alpha is a lower bound for internal consistency reliability). Second, I presume you and your colleagues have avoided obvious pitfalls such as including the same variable multiple times in a model.

Finally, and most important, please note that the admonition about CR being above .90 as undesirable presumes all constructs to be both fairly broad and (virtually) impossible to capture all relevant indicators. That may or may not be the case, but I doubt that it's universal. The thresholds you cite should be taken as guidelines, and not commandments.

Is it typical to have values so high (and this is what Christian Geiser addresses)? No. Is it impossible? Again, no. Is such a value proof of incorrect model specification? No. Does obtaining such values mean you should abandon your work? No. Should you validate your results with another sample? Always a good idea!. Doing so gives more confidence that your results are not sample specific, and therefore may generalize to the target population.

Good luck with your project.

Composite reliability is used to assess the internal consistency reliability of a reflective measurement model. It is deemed as an upper-bound estimator of internal consistency. According to Hair et al. (2022), its values should be between .70-.90—in exploratory research, .60-.70 is acceptable. Values of .90-.95 are undesirable, and values above .95 should be avoided. A very high composite reliability (i.e., above .95 as in your case) could point to semantically redundant indicators, which should be minimized (Hair et al., 2022). Here is the full citation.

Hair, J. F., Hult, G. T. M., Ringle, C. M., & Sarstedt, M. (2022). A primer on partial least squares structural equation modeling (PLS-SEM) (3rd ed.). SAGE Publications. https://us.sagepub.com/en-us/nam/a-primer-on-partial-least-squares-structural-equation-modeling-pls-sem/book270548

Good luck,

Hi David Morse!

"Second, I presume you and your colleagues have avoided obvious pitfalls such as including the same variable multiple times in a model."

Well, I think I did this mistake as I used redundant items for one construct. I basically have 3 items that ask pretty much the same question for the construct Data Protection Concerns. This is probably not exactly optimal.

Other than that, I realised that my constructs Performance Expectancy and Perceived Product Advantage are also quite similar. For my second version questionnaire I merged them.

I never worked in the field of PLS-SEM before and I guess I could have avoided those mistakes beforehand. But now it is how it is and I'm learning from them.. :-)

Thanks for your valuable feedback everyone! I really appreciate that!

Redundancy is not per se problematic. On the contrary, one of the most relevant concepts of classical test theory (measurement error theory) is that you can increase the reliability of test score (composite) variables by increasing test length, that is, by adding additional items from the same universe or population of items measuring a unidimensional concept. You can see this, for example, from the Spearman-Brown formula. The more (interchangeable) items, the more reliable the test.

There is nothing problematic about this per se, except when the redundancy of the items causes undesirable response behavior, for example, careless responding due to boredom, frustrating, etc.