I think B.

You can always reverse code items (or a similar linear transformation) in order to meet the requirements of A in regard to positive correlations within a construct, however, the correlations between such constructs could be either positive or negative.

One acceptable test of discriminant validity is to compare the Average variance extracted for each construct against the correlations (squared) between constructs. Hence, the sign in this test is irrelevant.

An additional point though is that the items are reflective of an underlying latent construct so a better measure of reliability is the correlations with the construct itself rather than between the item pairs. This is the argument for composite reliability over Cronbach Alpha's.

Of course it would be interesting to hear other perspectives on this as I am always learning new things by hearing from others particularly those in other fields.

Paul F Burke, thanks for your contribution. I'll put my cards on the table and say I agree with you. And you've made an important point by saying that items can be reverse coded if a positive correlation is desired (though that's probably a bit of a nifty sleight of hand that we need employ only if we're "cornered").

You mentioned the AVE, and I'll confess I wanted to avoid that, which I'd attempted to do by referring to nomological networks in the initial paragraph I opened this discussion with. What you have suggested is worth bearing in mind, however - I acknowledge.

Please forgive me if I don't enter into issues concerning interitem correlations, corrected item-total correlations, and coefficient alphas. Anymore, I regard each of them as a bit of a nightmarish quagmire, though, of the three, I am coming around to finding interitem correlations somewhat enlightening . . .

Like you, I'm interested to hear other perspectives. And I hope someone can come up with citations from the psychometric literature that are authoritative and appropriate. So far, citations have eluded me.

Thanks, again.

Hello Robert Trevethan ,

An interesting practical (and extremely influential) approach in this regard is Campbell and Fiske's (1959) multitrait-multimethod (MTMM) framework which offers relatively concrete and clear guidelines for examining convergent and discriminant validity based on MTMM correlations in a design that examines multiple constructs with multiple methods such as different rater types.

Campbell, D. T., & Fiske, D. W. (1959). Convergent and discriminant validation by the multitrait-multimethod matrix. Psychological Bulletin, 56(2), 81–105. https://doi.org/10.1037/h0046016

For convergent validity (e.g., of different raters/sources/methods with respect to the same construct/trait), ideally the correlations should be close to 1.0. However, already in 1959, Campbell and Fiske acknowledged that usually inter-rater and other agreement in the social and behavioral sciences is much lower than that. So they proposed weaker guidelines that were more comparative in nature (e.g., comparing monotrait-heteromethod correlations [indicating convergent validity] to heterotrait-monomethod correlations [indicating discriminant validity and halo biases within the same method]).

With regard to discriminant validity, I'd say it depends on the nature of the constructs. For example, for depression and anxiety measures, we would not theoretically expect those to correlate near zero for sufficient discriminant validity. We know that depression and anxiety are closely related constructs and often comorbid. So they would be expected to be strongly correlated (maybe .7 or .8) even if the measures are clearly distinct (one measures symptoms of depression, the other measures symptoms of anxiety). In other words, it would not speak against discriminant validity of the depression and anxiety scales if they were correlated .7 or .8. So it depends on what you're studying and your theoretical predictions.

One general issue with using manifest/observed-variable correlations for assessing convergent and discriminant validity is that they are attenuated by measurement error. We might thus underestimate convergent validity and overestimate discriminant validity when using correlations among observed variables. Also, the measurement error problem makes it difficult to compare those correlations for different measures (as proposed by Campbell and Fiske, 1959) because the measures may differ in their reliabilities. Some correlations might thus look the same even though they are different, whereas other look different but are really the same. That's why we now prefer models of confirmatory factor analysis and SEM for examining convergent validity, discriminant validity, and method specificity. These models allow us to separate true convergent validity and method specificity from random error. See, e.g.,

Eid, M., & Diener, E. (Eds.). (2006). Handbook of multimethod measurement in psychology. American Psychological Association. https://doi.org/10.1037/11383-000

Also of interest to you may be Fiske and Campbell's (1992) more recent perspective on these issues:

Fiske, D. W., & Campbell, D. T. (1992). Citations do not solve problems. Psychological Bulletin, 112(3), 393–395. https://doi.org/10.1037/0033-2909.112.3.393

Christian Geiser, thanks for your thoughtful and extended post. It's really refreshing to see contributions on an RG thread that have such substance.

You've set me off thinking about new perspectives as well as issues I've stumbled around in previously. For example, some time ago, I considered conducing an MTMM exercise (along the lines of the Campbell and Fiske '59 publication), and I simply gave up because of the complexity and lack of clarity concerning the results.

In some research I'm currently conducting, I have been sensitive - as you point out - to problems associated with reliability of the constructs we're using to assess both convergent and discriminant validity. I'm also concerned about the validity of the measurement of those other constructs. There are too many fundamental uncertainties.

In the field we're working with (teacher sense of efficacy), I'm also amused by other researchers claiming that similar correlation coefficients (usually in the .40s) demonstrate both convergent and discriminant validity - depending on what sounds best and, I suspect, on what might get their research published.

The only point I think we might disagree on is that in your third paragraph (starting with "For convergent validity") you refer to convergent validity in situations involving the same construct/trait. To me, what you refer to is certainly a form of validation. However, I regard it as different from convergent validity because I think of convergent and discriminant validity in terms of a nomological network, which I think is strongly related to what is aimed for in MMTM. I split up the two forms of validity.

Thanks, again, for your input. It's reassuring to have others share one's concerns and doubts.

Robert Trevethan, interesting! Campbell and Fiske (1959) defined convergent validity as "a confirmation by independent measurement procedures" (p. 81), and they operationalized convergent validity in terms of monotrait-heteromethod correlations (correlations between variables aimed at measuring the same trait or construct (e.g., teacher sense of efficacy) with different methods (e.g., teacher self-reports vs. student ratings).

I guess one researcher's convergent validity is another researcher's discriminant validity ;-). For example, a correlation coefficient of r = .8 for depression vs. anxiety might be interpreted as indicating high convergent validity by some (those are related constructs that are closely adjacent in the nomological net), whereas it might be interpreted as potentially insufficient discriminant validity by others (the high correlation may indicate that the constructs/measures are not sufficiently distinct). I think what is really key here is to have a strong a priori theory from which to derive very specific predictions and hypotheses. Unfortunately, in the social sciences, our theories are often not that strong and people do a lot of post hoc interpreting of results as they see fit...

Even Campbell and Fiske (1959) were fairly vague in their criteria for convergent validity (for good reason I think!) and only stated that the monotrait-heteromethod [convergent validity] values should be "significantly different from zero and sufficiently large" (p. 82; whatever "sufficiently large" means), that a "[monotrait-heteromethod] value for a variable should be higher than the correlations obtained between that variable and any other variable having neither trait nor method in common" (p. 82), and that "a variable [should] correlate higher with an independent effort to measure the same trait than with measures designed to get at different traits which happen to employ the same method." (p. 83)

Christian Geiser, thanks for pursuing this. It's obvious that convergent validity has different meanings or takes on different forms. One meaning refers to the context where the same construct is assessed in different ways - and I suspect that's the "monotrait-heteromethod" you're referring to - and Campbell and Fiske, also. You gave a good example of this kind of thing in relation to teacher sense of efficacy (TSE). (Thanks for entering "my territory" to do that.)

The kind of convergent validity I was referring to was, I think, closer to that of Cronbach and Meehl in which a correlation is sought between different constructs but that those different constructs are linked theoretically (within a nomological network) and are therefore expected to correlate with each other. Going to the topic of TSE, convergent validity might be pursued by seeking correlations of a measure of TSE with a measure of teacher burnout - with the theoretical expectation that those two constructs would be related to each other, albeit inversely.

For what it's worth, many researchers in the field of TSE claim that, theoretically, TSE is or will be related to burnout. However, there is only patchy empirical evidence to support the association between those two variables and I suspect that the measures of both TSE and burnout often have sufficient deficiencies to muddy the water.

It's all a bit complex, isn't it.

Maybe what I'm referring to should be called nomological validity in the way that is discussed in the following article:

Hagger MS, Gucciardi DF and Chatzisarantis NLD. (2017). On nomological validity and auxiliary assumptions: The importance of simultaneously testing effects in social cognitive theories applied to health behavior and some guidelines. Front. Psychol. 8:1933. doi: 10.3389/fpsyg.2017.01933

Robert Trevethan The example that you describe to me sounds a bit more like what in my field (psychology) would be considered "criterion [concurrent/predictive] validity" (does the TSE construct predict the relevant outcome burnout in a way that would be theoretically predicted in terms of the magnitude and sign of the correlation?) rather than construct (convergent) validity.

But, of course, in the end those are all just arbitrary terms anyway. Thanks for the reference--I'll check it out!

Christian Geiser, it seems we're both gnawing at these issues. Thanks for persisting.

Actually, I'm in the same club as you: psychology. About 13 years ago, I wrote an article that included information about construct validity (it was published in a top-level podiatric journal; I have since put a "replica" onto RG), and I was quite proud about the amount of thought I'd put into trying to sort out and write about psychometric aspects of scale development. I still think it is a pretty good article, but I'm now realising I could have done better.

For what it's worth, I've just discovered Chapter 9 in the following book:

Furr, M. R., & Bacharach, V. R. (2018). Psychometrics: An introduction (3rd ed.). Sage.

and I'm currently reading it with interest - particularly the first few pages of that chapter where convergent and discriminant validity are discussed in terms of nomological networks.