Hello Stephane,

Your question is an interesting one.  I am not familiar with the literature in your field, but I think that choosing "experts" should at least partly depend on who the instrument will be administered to.  Patients, in the traditional sense, for example, might not be regarded as experts, but if an instrument is intended to be completed by patients my sense is that they should be included among the experts used for determining face validity.  Ideally, instrument developers should have different groups of experts (some members from which could well have assisted in the creation of the instrument) and see whether there were different face validity perspectives from them. Perhaps one group would be sensitive to some aspects that another group of experts had overlooked.

I think researchers need simply to provide justification for the people who were chosen as experts for purposes of face validity - and that there is no definitive "group".

I hope that's helpful - but, again, I'm not sure what actually happens in your field - so I'm not really able to provide information regarding the hypothesis that you raise above.

Robt.

Good question. I think this shows one of the limits of face validity. Who are the experts?

If it does not seem "valid" for different groups of users (is users a better group than experts?), that does not mean it is not valid in other senses of the word validity (e.g., causal validity by Borsboom, Kane's how it is used validity, Meehl and Cronbach, etc.). One problem with experts is that they are not always right, This is why the US courts no longer use just general acceptance (Frye) for accepting expert testimony (the Daubert trilogy of cases addressed this). Galileo was not helped by "experts."

First of all, let us remember that the 'Face Validity' of an instrument (say a questionnaire) is simply that it may 'look like' it will return the data which it is intended to; but this may not be so (the questionnaire, for instance, may contain ambiguous word structures, leading questions, unfounded assumptions etc.---no matter who has created it, from PhDs to professors; let us also remember that such instrument 'glitches' are supposed to be detected in the pre-study piloting of any research instrument; therefore, one can only 'check' Face Validity in terms of assessing an instruments' worth or not for intended research; and the best person to do that is the researcher who is creating the instrument (in light of, where necessary consultation with other 'experts'---including intended end-users---in the manner suggested by Robert, above).

But the researcher who is creating the instrument is the prime-mover-----should have overall control.

Taking the questionnaire example: even after any necessary consultation with others---and of course, end-users (e.g. respondents) will always be necessarily consulted---the researcher remains in final control of this instrument; NEVER construct a questionnaire by committee (as committees harbour  the tendency to think that questionnaires are infinitely expandable---blind to the fact that such expansion equates to a useless data-collection instrument).

Thus, Stephane, I say that your hypothesis would only apply to dubious 'research'.