What can be good strategy when the quantitative and qualitative results differ in mix-method settings? Keep it in the same article or separate the results and prepare two different manuscript for different outlets?
The question brings lots of curiosity to know how is that possible and what was the context of the exercise.
As for your question, the paper will be more intriguing to show the contradiction and its justification, so put both results together.
Both the quantitative and the qualitative results need to be discussed in the light of the underlying philosophy of science that the researcher is promoting. This should be reflected upon. I agree - why is there a difference? Also - there could be difference between different studies using *the same* method - so this is not actually that strange. Two different application of the same method can give different result. Two application of two different methods can give different results. Not strange at all. Somewhere the researcher need to make a judgement which is *subjective* and this what all scientists make *human judgement* based on *interpretation* and *sense-making*. Choice of method was done by a human being, Choice of what variables were to be recognized as appropriate and which were to be recognized as not appropriate was also done by a human being, Choice of which question to ask and what scale or metrics to use is also done by a human being, All this and more is why *science* is to be recognized as some activities done *by human scientist*. All science has been done by scientists who interpret, reflect and make sense,
Muhammad - I do feel for your plight. Of the two mixed-method approaches i.e 'simultaneous' and 'sequential', it sounds like you have opted for the wrong approach. If you had chosen sequential, which is usually the 'safer option', then you would have avoided this. Ethically-speaking, if you started, conducted and got approval for this study as a mixed-methods approach - then you are more obliged to publish it as such within a single publication. On the other hand, in my mind, it is not wrong to publish findings separately - as long as each article identifies that 'it is part of a wider mixed-methods' study - but that 'these particular findings are more appropriate for this journal'.
Muhammad, what was the context of your research? Dean brought in an important observation, did you use a simultaneous approach?
Interesting topic! The intention of the Mixed methods research is: one method should complement the other method. The quantitative and qualitative studies do not measure the same, Therefore, their findings cannot be reported in the same manner. Generally, the findings of the quantitative can be made sense through your qualitative findings, but, not always. There could be something that is not clear enough, then it is worth exploring it further.
I did come across two paper (by the same author) that had contradictory findings and were reported as two different papers. They had measured the impact of an intervention both quantitatively and qualitatively. To me, that was the error. You cannot measure the impact of an intervention qualitatively, but explore the perceptions of the participants. Luckily, I got to read both of them and could draw my own my conclusions which were not in agreement with the authors' conclusions, but, were important to me.
I am very curious as to learn a bit more about the methodology and the research questions, so I can give a better opinion. My email address is [email protected]
I agree with most of the comments and especially the best way to use qualitative research is to obtain pilot data on the population(s) you wish to study, the best way to categorize and ask survey questions to answer the most pressing issues exposed through the qualitative phase, and therefor my own mixed methods research uses the sequential approach to help in the design, sample recruitment, quantitative data collection and the analysis and interpretation of the the findings from the quantitative data. Perhaps Muhammad was really more concerned about two different mixed-method studies within the same population and looking at similar issues that produced conflicting results. In that case, if possible, re-interviewing participants in the two studies to see if their understanding of the key questions being asked differed between the studies and how the wording, administration and multi-item scale development might have contributed to the disparate findings between the studies.
We know from the long and rich history of survey research that many factors can influence the ways in which respondents answer similarly worded questions, particularly in the area of "sensitive behaviors" such as drug use, sexual behavior and even adherence to an experimental study condition/treatment. We also know that the verbal and non-verbal communication that may take place between the examiner or clinician describing the study to participants can greatly effect the perceived effects of control or placebo conditions. While it is still impossible to predict and control for all of the potential conditions and question wording and administration that might significantly skew the results of one study vs. another, I think that the increasing importance of mixed-method approaches to new and emerging health problems can go a long way towards minimizing differences between study results and to explain the residual differences between study findings that warrant further investigation.
This sounds like it goes back to the classic use of two different methods to produce "convergent" findings, i.e. getting essentially the same results through sources that have different strengths and weaknesses. One of the reasons that this approach has become less frequently used is because the results often diverge, rather than converge, in important ways.
I think this potential for divergence is easy to understand, whenever one is studying a reasonably complex human phenomenon. In particular, the quantitative methods could well emphasize one aspect of what you are studying while the qualitative methods emphasize a different aspect. So, it is not necessarily the case that one is right and the other wrong; instead, they may each be capturing different aspects of what you are trying to study.
With regard to publishing, my recommendation would be to put the two into the same paper and to emphasize the divergence from the beginning. The goal would thus be to make the different results into an interesting and challenging problem for your field to consider in its future research.
Mixed method is an emerging promising trend in social science research and emerging research may divert their research design towards this in order to avoid decade long debate on quantitative vs. Qualitative....
Good response David. Good reviewers/editors will celebrate 'divergence' on the basis that the findings are honest and identify multiple outcomes. Some authors, however, may well be hesitant about submitting manuscripts that report divergence, rather than convergence; thinking that the findings might be construed as conflicting.
I agree that the divergent findings should be kept in one study. If the study is solid and properly executed the editors and reviewers may even consider the disparity in the findings as a strength rather than a weakness.
In this situation, the researcher can provide and engaging discussion as to why the results diverge and are not such as was expected when the study had been initiated. Perhaps, the researcher can get a good insight concerning what to look for in the next study and give valuable advice to other researcher for future research vectors (perhaps, adding some extra / novel variables in the QUAN strand?).
Muhammad Mohiuddin
Your question has a potential to drive the research forward! Although such contradiction seems to stuck the work, it actually points at how rich the methodology can be, and helps finding deeper insight into the scientific problem.
Just to report two examples from the scientific literature:
1) Tonkin-Crine et al. (2016) Article Discrepancies between qualitative and quantitative evaluatio...
report that "a triangulation protocol (i.e. a protocol to integrate data in mixed methods research) can also enhance the validity of findings and assess whether data agree (convergence), complement one another (complementarity) or contradict each other (dissonance) [1]. Dissonance, in this respect, does not indicate a failure in the study but can be considered constructive if it leads to new findings or a richer understanding [3]." The authors further give and discuss an example of such dissonance.2) Wagner et al. (2012) Article Reconciling incongruous qualitative and quantitative finding...
pointed out that "cases in which findings from different methods are congruous are generally thought of as ideal, while conflicting findings may, at first glance, appear problematic. However, the latter situation provides the opportunity for a process through which apparently discordant results are reconciled, potentially leading to new emergent understandings of complex social phenomena." The authors discuss three studies presenting apparently discrepant results.
Thank you everyone, I am learning just as much as Muhammad if not more.
In a nested study both quanti and quali are done at the same time i.e measuring intervention and exposure at the same time for quanti (and perhaps interviewing the same people in the quali). Here anything goes or rather expect the results to converge or diverge. Expectation usually is to triangulate the quali with the quanti to get a comprehensive study.
If it was possible to, do the quali on a different (homogeneous to your sample) set of participants (or do it earlier on the same participants). This way you have kind of piloted the study, you could then revise the data collection tool if need be.
It would be good if your study was theory driven, then you could uphold the theory assumptions or develop it further. I mean this would be great Muhammad.
In your case you might consider employing a Collaizzi approach of going back to the same participants after some time. You will need to take care of reflexivity now that you know of the divergent results. A study like this one is great in my opinion - since it has potential to generate lots of theoretical inputs and can go on and on...
Concerning publishing it as of now, I would publish them as two different studies for now, though I see there are more qualified experts (with due respect) who have said otherwise here.
For posterity I have learnt something that will ever need to be acknowledged as a limitation of (should we say for all? mixed studies) - the possibility of divergent results. This will need to be declared in the proposal. I will be doing that just now.
For me, I intend to do a Delphi survey on experts in nursing curriculum, do an indepth key informant interview with nurse managers, but do a survey with the practicing nurses. I know it will give a richer view on the phenomena 'reorienting nursing curricula to universal health coverage'.
If I get divergent results I will perhaps be disappointed (to some extent) but I will not be surprised. I am watching this space.
The only case where I got different findings using either qualitative and/or quantitative research is in small samples which I solved by looking into exact analysis, there Pearson's R may switch results versus Spearman. On the other hand, I always used mix methods to complement each other. Reporting divergent results is done in the same paper as also mentioned by colleagues here.
Currently I am working on a paper where I am running Qualitative (nonparametric) versus Quantitative (parametric) tools to compare outcomes.
‘Complement’ (cf. Hussin Jose Hejase ).
We must not get lost in this qualitative/quantitative dichotomy. Just think of the space of the phenomenon and draw tools of both persuasions. The real challenge is courage. We must admit lack of access to certain tools. Researchers who admit their weaknesses produce the more useful cases. Weaknesses admitted allow other researchers to look at the same phenomenon from a different vantage point and with new tools. Those investigations will trigger new questions. At the end, the insight of the questions are all that matters.
As I am learning a bit more about research methods, I have read that researchers often use qualitative and quantitative material to complement each other. A third way is to design a study where qualitative and quantitative data are collected and analyzed at the same time. Results from one method can be extended or triangulated by using another method.
Qualitative methods use coding techniques to organize and categorize data into meaningful divisions for further analysis. Subjects, such as leadership effectiveness, may call for using quantitative and qualitative methods. Use of the two methods together is called a mixed methods approach to research.
The divergent results should be kept in the same paper. In my opinion divergent results of mixed methods research, give the researcher the chance to explore more the phenomenon under study. Such findings should enriche the discussion and may emphasize the need for further future research in the area of interest.
Certainly, divergence of results is very good and should be part of one paper for two reasons. First it stops people from "forcing" results of the two methods t be the same. Second, by submitting divergent results, we open doors for future research in the area thus providing more insight and enriching the study area.
Divergent findings in a mixed study should be reported in the same study particularly in studies involving human research. This will add flare to the study and bring out the true variety of human nature. it will also prompt further research in that area.
- Badges
- Science topic
- Similar topics
- Philosophy
- Esthetics
- Interpretation