Hi, Martin,

Important question--the condition you describe applies to many frequently-used "measures of constructs".

Researchers should NOT use an inferior (psychometrically weak) measure of a construct--garbage in, garbage out.

However, this does not mean the construct should be abandoned, but rather that measurement of the construct should be improved.

Best regards...

Hi martin,

It is a good question.

The answers is beyond statistical analysis in the sense that the construct that you might be referring should be theoretically sound. This means that certain physical or other theoretical principles must be respected. So, for instance, mass cannot be negative. Although many constructs are empirical: namely, statistical analysis aids the formulation of the construct. Nonetheless, generalizations should have a theoretical basis or else they will be limited to the sample (and the population) from which the sample was taken. For inference, from the sample to the population, you will have to assume homogeneity, same behaviors, and so on. In your case, although I do not know it, the information you have generated supports disconfirming your initial hypothesis. But, it does not mean that the construct is incorrect. Consider the following.

Diminishing marginal utility of income (dmui) is theoretical construct that can be tested via statistical methods. You may get statistical results that disconfirm it. Should you the conclude that diminishing marginal utility of income is incorrect? I suspect most people would say that the empirical results do not disconfirm the construct. So, look at the literature: you might find that most experiment have confirmed what you thought may be true. And yet, if the population from which the sample was drawn (at random) has an underlying behavior that counters the dmii, then what would you do.

Hope this has helped a bit. You have a really interesting problem. try meta analysis of the literature that regards your issue.

All the best,

Paolo