Dear Sa Rehman,

Rigor in a qualitative study is achieved through employing correct and established procedures for data gathering and analysis. There are several well-written articles on the correct procedures and I am sure you must have read a lot of them already.

Triangulation is the answer to both the questions that you have asked; regarding employing multiple methods and rigor. Triangulation involves interviews, observations, focus group, member check and other methods that ensure that the data collection is correct and consistent. I see that you are conducting semi-structured interviews and studying the policies. Perhaps you would also want to conduct focus group to get a deeper insight. Also ensure that member check is employed.

I believe that if you do all these, the issue of rigor will be taken care of. I would strongly recommend you to read articles written by Guba. Reading Silverman might also help you.

All the best.

Silver man and Guba

 Dear Noman Mohammad,

Thank you for answer. I suspected that my methods might have been leading to triangulation in a qualitative methodology to be honest. Does it matter that I still see myself as a pragmatist? Or is constructivism more accurate for this purpose? 

P/S: Thank you for the suggested readings as well. 

Well, as per my understanding being a pragmatist or a constructivist (or both) is not a major issue. The basic difference is in your epistemological assumptions; you are a constructivist if you try to uncover meaning from the data you have collected with YOUR viewpoint playing an important role. On the other hand, you should call yourself pragmatist if you intend to find solution to your problems by collecting data.

Hi, Sa,

I recommend you to read this great article about rigour in QR: https://xa.yimg.com/kq/groups/73868647/750861395/name/Trustworthypaper.pdf

Hi Rafael,

Thanks for that great article! It'll be useful for my end of year review!

Your welcome, Sa. I have more references if you are interested.

If you are indeed doing triangulation, then I don't think you need to worry about whether your quantitative study is placed within a qualitative one ("embedded"). The key issue in triangulation is whether independent qualitative and quantitative studies yield the same results.

If the results from the two studies converge, then you have more faith in both of them. But if the two results are different (i.e., they diverge) the situation becomes more complicated.

Also, the sources of data that you describe do not limit you to triangulation, In particular, you could do a sequential study of the form QUANT --> QUAL. This would begin with an analysis of the quantitative data, then a follow-up qualitative study to pursue those results in more depth and detail.

Hi David,

Thanks for your answers to both of my questions. I think at the moment I would have to say that my research is leaning towards multimethods as advocated by Morse (2003) with a QUAL+qual formula. Doing a mixed methods would require me to come up with a quantitative design. At the moment I don't think I have such a solid quantitative study which would make my research design cross the paradigms of quan + qual. 

I've changed my research design and now it's a multimethod QUAL + qual using document analysis and a case study. I've chosen this to ensure that my research is much more rigorous.

Could you please comment on these changes? I could really use some academic advice, thank you in advance. 

If you begin with a major document analysis, and then use that as the basis for choosing a case study, that sounds sequential to me: QUAL --> qual. The key issue is the extent to which the design for the second study depends on the results from the first. If you don't have that kind of follow-up connection, then it would be QUAL + qual.

Going back to your earlier mention of the existing survey dataset, you could do secondary analysis on that data as part of a mixed methods study, so long as you weren't repeating things that had already by analyzed in the data. 

FYI the current issue of Qualitative Health Research (edited by Jan Morse) is about mixed and multi-method research that emphasizes the qualitative component of the project.

I will put more thought into the timing of the data collection/analysis. 

Thank you David, you have been so kind and helpful! I'm looking at the journal at the moment and you have been a true lifesaver today! Thank you again. Best wishes. 

Hi Sa Rahman,

In addition to semi-structured interviews and document analysis, you can do a focus group (e.g., with the 'experts') to further discuss what you have already found from your earlier field research, and to refine & reframe your conclusions and policy proposals. If you recruit the 'right kind of participants' for this purpose (i.e., diverse and knowledgeable experts wirh diverse sets of approaches), it might also help you as a fact-checking and proof-reading process:-)

Good luck with your research & reporting!

Dear Sa,

I have been dealing with some kind of your issue when I was conducting research for my dissertation, except I'm not a big fan of quantitative methods so I refused to do them (which applies also for mixed methods). Your dilemma has, I believe, several levels. First, you should think of data creation - what documents or textual production you will be analyzing and with whom you will be conducting the interviews. You should focus on your research problem in this regard. For example, I suspect no government officials will tell you about an economic situation in household, simply put, they don't know it and they will be presenting their mental maps (probably some media frames). Households or social workers (state or NGO), on the other hand, are something completely different. In the first case you will gain some cases with very detailed information, and in the second, beside the cases and information, also reflections of, say, governmental policies, which is exactly what you need. So it's good to remember that different actors have different information. The similar applies for documents and textual production - what document will you collect (again, what actors)? Or will you collect only documents, or also media production (for example, of politicians presenting the problem etc.).

Second, as for mentioned triangulation, it is, in some way, automated process during the analysis. In your case it will be definitely part of your analysis, whether you want or not :). Let me give an example of triangulation in order to clear things up. Triangulation is used every time when several sets of data are created and analysed within one research. Your problem is how macro-economic changes affect households so you have some statistical data (if I get it right) and you will be conducting interviews as well as document analysis. These are three data sets altogether, which means you have three different data sets that should ideally point to the same main issues, albeit in its own way. So, for example, you will identify specific macro-economic affection in statistical data, this affection should be present in interviews with households (at least in some of them), and the same affection should be mentioned in the documents. Or every data set can point to somewhere else and you even discover some new affections and connections. The point of triangulation is in comparing the different data in order to have greater variability and credibility of your research. Of course, researches done before yours can be also regarded as relevant data set for triangulation.

Third, you should think about the analytical and explanatory theories since they are intertwined with research problem and data creation methods. Keep in mind that the qualitative data creation is linked with qualitative data analysis and that for analyzing the statistical data you will have to use different theories. The comparison of different outcomes of data analysis can be sometimes also difficult task.

I hope you will find any useful ideas or incentives in the answer, and good luck with your research :).

I will be brief. By definition, all qualitative research usually use diversified methodologies. And second, the methodology you are going to use depends first, on the research question you have and second, the conditions of the fieldwork. In other words, "the effects of macro-economic changes to households" is not precise enough to define what type of qualitative methodologies you really need to develop. If I understand you well, you originally planed to do a quantitative research, but since it does not work anymore, then you are moving back to something more qualitative. I am not sure this is the right approach. To me, the right approach should be to ask yourself how the existence of a new set of data affect your research (question). Second, what do you really want to answer. And third how you are going to answer it. The last point would be your methodology. Then, you can choose semi-structure interviews, recording of "life stories" (followed or not by a content analysis), focus groups, filming, participant observation -- or a mix of all that. Most of the time, there are no "typical" qualitative methodologies,  but the researcher needs to be a bit imaginative to see how to answer his research question. I hope this helps.

I agree with Louis's response and have the following additional comment for your consideration.

If you have good secondary data that you can depend on for reliability and validity, Could you perform a content analysis on the secondary data to draw some preliminary conclusions that are then evaluated in your qualitative work.  Even though secondary data is available would you not want to quantitatively measure your sample population to understand the relevance of the issues within this group and then further explore in your interviews? In my dissertation I am using a mixed-methods (quantitative-qualitative) sequential explanatory study following the methodology in Ivankoval, Creswell, and Stick (2006). 

Ivankova, N., Creswell, J. W., & Stick, S. L. (2006). Using mixed-methods sequential explanatory design: From theory to practice. Field Methods, 18(3). doi:10.1177/1525822X05282260

Have you looked at Bryman's tome on research methods? The latest edition should be helpful if you haven't seen it yet. 

I've spoken to my SVs about it and have decided to use Morse's definition of mixed methods... Mine will be qualitatively-driven. I will apply semi-structured interviews + content analysis + document analysis. Thanks for all of your answers, they have been really helpful :)

I might suggest that as part of your semi-structured interviews you collect the same data you were going to collect in your survey, or the same data found in the new data source to which you are referring.  While not statistically significant, you will be able to tell where individuals fall in relation to the larger population, it will also help you to evaluate the qualitative data collected to place this segment of the population you interview withing the larger context.

I dont necessarily see a problem in collecting both quantitative and qualitative data.  Even if primary data already exists, it does not always focus specifically on your exact research needs/ question(s) etc. In some of my research, Ive collected reasonably large scale survey data in the first instance, and  then followed up with indepth semi structured interviews.  It all boils down to your research questions and how you relate the quantitative and qualitative data collection/ analysis etc

Dear SA,

have a look, if the following publication helps.

Best wishes,

Nina

Article Mixing process-generated data in market sociology

try this paper:

A Comparative Analysis of Qualitative and Quantitative Research Methods and a Justification for use of Mixed Methods in Social Research

Rigorous research means carefully devising data collection and analysis strategies that will be useful for answering your research questions. So the questions for you to consider are, Do you have important research questions that quantitative data from interview participants would help answer? Or would people's responses to your semi-structured questions provide more useful information for the purposes of your study?

Also, rigor is determined less by the type of method than by choosing the right methods and how the researcher carries out that method. For example, an interview study can be made more rigorous by: interviewing more people in the population of interest to make the findings more generalizable, transcribing the data to ensure that all comments are captured, conducting multiple rounds of coding to increase coding completeness and accuracy, having more than one researcher coding the data to minimize researcher bias, etc.