How can I quantify construct validity without a gold standard and where a new measure is not put forward, but old ones are being compared?
Dear Dean,
This is a rather complicated question to reply to. Construct validity requires the comparison between two scales, one being a gold standard. However there are ways to bypass this, although generally, it's not advisable.
In my opinion, I believe the answer to your question will only be evident after you have conducted the study. The results of your study should have some resemblance to previous studies. In that way you could say that the results tally with previous existing results.
A word of caution though. The study has to match the criteria set in the previous studies to facilitate some form of comparison. Hence if you are trying out the questionnaire on obese patients, it would only be fair if the previous studies you wish to compare utilized the same study population.
Secondly, you could do a confirmatory factor analysis to prove that your factor structure fits the findings found in previous studies.
Hope this helps. And good luck in your research work!
Rabin
I'd agree with Rabin's answer. One other possible option (although this skirts the borderline between construct validity and criterion validity) would be, is there any behaviour or other external attribute that you could measure for your sample, perhaps by using observers or by ratings from peers/friends/family, that demonstrably relate to the construct you are trying to measure?
A gold standard is not necessary for construct validity. For some measures, there are no gold standards. Therefore ,you could use triangulation of validation. Meaning you use several techniques of validation. To assess construct validity you could perform Exploratory Factor analysis or confirmatory factor analysis. If you are measuring a previously developed instrument/ tool you could use confirmatory FA. If this said tool/ instrument is being used on a different population you have to adapt and then validate it.
Hope my answer helps
Maybe the method described in the following article is helpful: Moore & Benbasat. 1991. Development of an Instrument to Measure the Perceptions of Adopting an Information Technology Innovation. Information Systems Research.