Dear Eliseo

To be clear, we are talking here about ‘instrument validity’ not ‘study validity’ (internal or external).

If we take one of the classic social methods discussions of instrument validity, they would suggest that there are five ways to validate an indicator (for instance, a question asked as part of a questionnaire, or for that matter during a qualitative interview).

a) Face validity: to claim face validity, you normally rely upon previous studies that have shown that this is a good indicator or measure of the thing it is meant to measure. An example would be an exit poll: it has been shown that asking people how they just voted in an election is a good measure of how they did vote (not many lie).

b) Criterion validity (concurrent): this is used if a measure does not have face validity in its own right, by checking it against something with face validity. For instance, if you ask ‘how far do you have to walk to work?’, you can check the validity of what people answer by checking a sample of responses in your pilot with a pedometer to check the actual distances walked.

c) Criterion validity (predictive): like the previous example, here you need some outside validation, in this case in terms of whether an indicator predicts behaviour. If you want to ask people about their help-seeking behaviour during illnesses, you could check the validity of your question by seeing how many times they visit the doctor over the next year.

d) Construct validity. This is used when there is no face or criterion validity test, and is based on a theoretical or conceptual justification that a measure has validity. For instance, it may be logical to assume that holding certain progressive attitudes on current affairs will predispose someone to vote for a political party known for supporting such causes. Eliciting respondents’ views on these topics can be argued to be a good proxy measure of future voting behaviour. This kind of theoretical validation is often used by market researchers, who know that it is hard to get people to give honest answers if asked straight out ‘how are you going to vote?’

e) Finally, you can also use ‘content validity’: this is achieved by using a range of measures that are accepted (face validity) as covering the main features of a phenomenon. For instance, you can find out about a person’s social class if your questionnaire covers income, education, cultural pursuits and attitudes, which are accepted as together comprising social class. This kind of validity is often claimed for questionnaires and interviews that ask a wide range of questions about a topic.

So you can use any or all of these different methods to validate both quantitative and qualitative research instruments, including interviews.

Does this help?

Nick

Eliseo,

Questionnaires (which have to be unstructured or semi-structured in qualitative approaches) are best not viewed as 'instruments' in the sense of structured quantitative questionnaires - but more as 'guidelines'. The more structured a qualitative questionnaire is - the less likely that a 'natural' free-flowing narrative event will occur. The term 'validity' also is more a term to apply to quantitative than qualitative approaches. 'Trustworthiness' is a more appropriate context to apply. To 'measure' the trustworthiness of a qualitative questionnaire schedule - the best approach is a team expert review of the questions themselves - for reducing ambiguity, leading questions, emotive questions, stressful questions etc

Thank you very much dean for an informative answer. How I wish I can keep in touch with you regarding matters on research.

Good day and God less always.

I agree with Dean, but another way is to perform a reliability check by comparing measurements such as the carrying out of some three focus groups and compare for fashion or repeat constructs

I agree that validity is a quantitative concept, and I try to avoid using ideas that are about one type of research as standards for assessing a different type of research.

Still, analogies can be helpful, as long as they aren't taken too far. In particular, one of the key validity questions with quantitative questionnaires is whether you are actually measuring what you hope to measure. Or, put another way, are the survey respondents thinking about the same things as you are when they answer your questions.

The analogy for a qualitative interview would be a goal of hearing about what you want to learn about. I like to summarize the goal in qualitative interviews as "listening and learning," so you need questions that will guide your research participants to talk about things that will help you understand their lives and experiences.

Finally, in terms of standards for qualitative research, I like to reason back from an ultimate criterion of "credibility." You want to do your research in a way that others (e.g., your readers and reviewers) will find it to be both believable and meaningful. So, what kinds of research procedures will help you meet those goals? That idea also reinforces what Dean suggested -- that hearing how others react to your draft set of questions will help you interpret whether or not those questions will serve your purposes.

Dear Eliseo

To be clear, we are talking here about ‘instrument validity’ not ‘study validity’ (internal or external).

If we take one of the classic social methods discussions of instrument validity, they would suggest that there are five ways to validate an indicator (for instance, a question asked as part of a questionnaire, or for that matter during a qualitative interview).

a) Face validity: to claim face validity, you normally rely upon previous studies that have shown that this is a good indicator or measure of the thing it is meant to measure. An example would be an exit poll: it has been shown that asking people how they just voted in an election is a good measure of how they did vote (not many lie).

b) Criterion validity (concurrent): this is used if a measure does not have face validity in its own right, by checking it against something with face validity. For instance, if you ask ‘how far do you have to walk to work?’, you can check the validity of what people answer by checking a sample of responses in your pilot with a pedometer to check the actual distances walked.

c) Criterion validity (predictive): like the previous example, here you need some outside validation, in this case in terms of whether an indicator predicts behaviour. If you want to ask people about their help-seeking behaviour during illnesses, you could check the validity of your question by seeing how many times they visit the doctor over the next year.

d) Construct validity. This is used when there is no face or criterion validity test, and is based on a theoretical or conceptual justification that a measure has validity. For instance, it may be logical to assume that holding certain progressive attitudes on current affairs will predispose someone to vote for a political party known for supporting such causes. Eliciting respondents’ views on these topics can be argued to be a good proxy measure of future voting behaviour. This kind of theoretical validation is often used by market researchers, who know that it is hard to get people to give honest answers if asked straight out ‘how are you going to vote?’

e) Finally, you can also use ‘content validity’: this is achieved by using a range of measures that are accepted (face validity) as covering the main features of a phenomenon. For instance, you can find out about a person’s social class if your questionnaire covers income, education, cultural pursuits and attitudes, which are accepted as together comprising social class. This kind of validity is often claimed for questionnaires and interviews that ask a wide range of questions about a topic.

So you can use any or all of these different methods to validate both quantitative and qualitative research instruments, including interviews.

Does this help?

Nick

Good, measured response Nick!!

Qualitative research has parallels to the quantitative research goals of validity and reliability. In qualitative research validity argues for a kind of rigor in the application of method, whereas the reliability argues for both community consent and a form of rigor.

Guba and Lincoln (2005) have analogous criteria for qualitative validity, which are authenticity, trustworthiness, fairness, ontological authenticity, tactical authenticity, catalytic authenticity and educative authenticity.

Reliability was addressed through detailed notes, a good quality tape recording, and by transparency in note taking.What is important is the quality of the research.

Lincoln (1995) identified eight standards of evaluation in qualitative research: positionality (honesty, authenticity), community (serves the purpose of), giving voice to multiple voices, critical subjectivity (heightened self –awareness), reciprocity, sacredness of relationships in the research-to-action continuum and sharing of privileges (cited in Creswell, 2007; Guba & Lincoln, 2005).

Yardley’s too has criteria which are sensitivity to content, commitment to rigor, transparency and coherence, impact and importance (as cited in Hefferon Gil-Rodriguez, 2011; Smith, Flowers, Larkin, 2010).

Konrad - another good response. I agree that parts of what you identify will assist in qualitative rigour. However, to try to implement all, or even good parts, of the processes you raise - would probably constrict and constrain the activity. Cherry-picking a few aspects you suggest would certainly assist though. That's why earlier I only raised the point of trustworthiness which, in itself, has several components to it.

Hi Eliseo,

There are ways that has been suggested to ensure validity - accuracy of the research findings - when using qualitative methods. A good starting point to this is this reference - Creswell, J.W. 2007. Qualitative inquiry and research design. 2nd edition. London: Sage Publications Ltd.

In that book, Creswell has proposed ways of ensuring accuracy of the research findings. Some of them are:

a. Prolonged engagement and persistent observations in the field - having as many interviews and discussion with the participants as you can,

b. Triangulation strategies - having many sources of data using several data collection methods and tools,

c. Producing rich, thick description of phenomena you are studying, and

d. Respondent validation of preliminary findings - going for feedback to the research area and getting the feedback from the research participants.

Please read the book for further understanding.

Cheers.

Thank you. This discussion is helpful. Permit me to quote your explanations to strengthen the discussion on this topic with my students. 

God bless!

Thank you all, for what has been addressed, in fact my concern is that, do we have a seperate methods for the quantitative questionnaire?

Hi

Regarding the validity, several remarks need to be incorporated:

1.Still, this unified concept of validity is best understood and examined within the context of its four discrete facets: content validity, construct validity, criterion validity and consequential validity.

2.Several authors suggest employing the following four steps to effectively evaluate content validity: 1) identify and outline the domain of interest, 2) gather resident domain experts, 3) develop consistent matching methodology, and 4) analyze results from the matching task.

3.Several series of steps in which to follow when pursuing a construct validation study:

1) generate hypotheses of how the construct should relate to both other constructs of interest and relevant group differences,

2) choose a measure that adequately represents the construct of interest, 3) pursue empirical study to examine the relationships hypothesized, and 4) analyze gathered data to check hypothesized relationships and to assess whether or not alternative hypotheses could explain the relationships found between the variables.

4.Criterion validity refers to the ability to draw accurate inferences from test scores to a related behavioral criterion of interest. This validity measure can be pursued in one of two contexts: predictive validity or concurrent validity. In predictive validity, researchers are interested in assessing the predictive utility of an instrument.

5. For more information, please refer to,

Instrument Reliability and Validity:

Introductory Concepts and Measures

Amanda Jane Fairchild

James Madison University

Regards,

Zuhair

Please refer to this wonderful book!

Fraenkel, J. R., Wallen, N. E., & Hyun, H. H. (1993). How to design and evaluate research in education (Vol. 7). New York: McGraw-Hill.