As part of my thesis, I have gathered data from a survey from 300 individuals from various automotive divisions using sequential mixed methods (quan to qual) and then interviewed 11 individuals to address some questions from the quan findings. I had problems in interviewing any of the 300 individuals for the following reasons:
1) They do not want to have their views documented. They refused to be interviewed
2) They fear they could lose their jobs over the comments.
3) Their english is poor.
As a results, I had to interview 11 middle management who have more than 20 years experience in various divisions but they are not part of the 300 individuals who participated in the survey.
My question is as follows:
1) Are these comments considered acceptable reasons? How do I put these into acceptable comments suitable for academic purposes?
2) Is the interview with the 11 middle management employees acceptable?
3) What are your views on the above approach?
Hi Tze Leong,
The first thing that came to mind when reading your question was that unless there is a direct link or relationship to your study question to the responses from middle management, then it would be hard to make a strong case for using this sample for the qualitative portion. Maybe you could tell us what your research question is so we could understand the appropriateness of the sample? Because unless you set out to compare and contrast two different populations, in which case you'd still have to interview some of the initial participants, then I'm not seeing a defendable reason to switch samples with the information you've provided.
Additionally, what struck me was the fear in the responses you got from some of the initial participants. I'm assuming all participants were given full anonymity to avoid such repercussions. Furthermore, their responses to being interviewed could stem from cultural differences and not trusting you enough yet. What sort of methods have you used to establish trust, transparency and honesty with this population? Would they be more likely to talk with someone who speaks their language, etc.?
Good science, in my opinion has to establish trust and honesty between the researcher and participants before you'll get the candid and expressive forms of data from interviews that you need in mixed methods. Try re-examining your recruitment materials so they provide the type of information that a prospective participant might wonder about. Put yourself in their shoes and think about what you would need to hear before revealing personal information to a stranger. For example, where are you planning on reporting the results? Who has access to the data? Who is the audience? Are you seeking to publish the results at the academic level or will they be used in a practical sense to improve business systems? What methods were established to keep the information confidential so there could be no ramifications for the participants to come forward? Etc….
A possible work around would be to be upfront in your thesis (and your chair) and state that due to the unexpected responses you received in the second phase you were unable to do the interviews within the time frame allotted for your thesis. As a result of this limitation, during the analysis portion of your study you had to switch to a straight quantitative design in order to utilize any of the data. This would allow you to at least finish the project within a specific time frame if you are limited on time and resources and can address these issues in your discussion section.
Hope this helps!
Dear Angeline
The research question was about how innovative work behaviour could mediate the relationship between mindfulness and performance in workplace. These 11 individuals are a mixed of managers and executives. I considered them as middle management because there are 3 general managers and the balance are managers and executives. Since the sample for the survey had almost 80% workers and 20% executives, it could still be argued that the 11 individuals could still represent the views of the survey due to their extensive experience in the various divisions, they were previously supervisors of these workers and higher saturation level of the comments. Also, the survey on innovative work behaviour and performance was based on the employees' supervisors perception and only the mindfulness part was completed by the workers.
I agree that culture and trust were the main reasons. Full anonymity was given and yes, I did try to establish trust. It was not a matter of speaking the same language. Once the idea that excerpts of their conversation would be published, the resistance to participate was there. They are shop floor workers who do not have degree and most of them work from a young age. To trust someone with their rice bowl is is difficult and I cant' really blame them. My Phd is an industrial PhD and my employer is aware of my studies. It is possible that they may want to look into the findings and hence my dliemma in how to frame the appropriate response in my thesis. As outsiders and academics, my panel of examiners were appreciative of the quality of the feedback. However, they suggested me to include the reasons. The problem is how to put in an appropriate response that meets the academic and employer perspective.
I agree that the relationship of the two samples to the research question is crucial. It sounds to me like they are indeed both relevant, and so you would be comparing the views to people from two different levels within the same industry.
For that purpose, doing a survey is a good choice for representing the large number of employees, while doing in-depth interviews is a good choice for hearing from the limited number of managers. For the workers, your data has the strength of widespread coverage, while for the managers the strength is that you are hearing things in more depth and detail.
From a qualitative point, learning that your survey respondents were not willing to do personal interviews is basically a research "finding." In qualitative research, this issue is known as getting access to the desired informants, and there are all kinds of reasons why you may not be able to get access. So, you have learned something about the organizational culture of your research site -- especially when you compare this to the willingness of managers to participate.
An examination of the methodological assumptions is always important for the structural analysis of science. The choice of method is the criterion for distinguishing disciplines. The classical philosophers looked at the results. Today we look at the path or route that leads to the truth. The emphasis of the research gradually shifted on the route ahead and ran and finally to the ways of conducting research. On scientific methods and their variations influenced not only the experiences of research, but also the philosophies, particularly positivism. This gained a hegemonic position first, then declined inexorably. Epistemologists and philosophers agree that "the progress of science and the crisis that began in the math, with the non-Euclidean geometry and the new abstract theories, then physics, thermodynamics with the theories of relativity and ‘quantum’, have shown the inadequacy of the positivist methods and the need for a critical theory of science and the scientific method.
After this introduction, I enter into the argument by saying it is not my intention to reduce philosophy to pure methodology. First of all, fallen the fundamental postulates of positivism, the proposals for the scientific method or inventive research and discovery of reality increased: the systematic arrangement of knowledge, then came: descriptive, genetic, historical, normative, axiomatic, phenomenological, dialectical and structural. Of the last three is the philosophical matrix. they tried to sort the various operations: observation, description, experiment, analysis, definition, demonstration, classification, explanation, analogy, hypothesis, postulates, and theories and so on. These redundancies denounce the difficulties.
The problems that our research team has faced show that the relations of knowledge and scientific theories with reality will never be definitively resolved, but will remain open to always new insights and reflections. It is now clear the complexity of the issue. Scientific theories are not reducible to mere mental exercises or structures without contacts with reality, as claimed by the conventionalist. They are not even comprehensive and definitive pegs with reality as claimed by the royalists. They include, in fact: a) elements derived from the more general context of life, culture and society; b) general paradigms stemming from such circumstances, but constituting the background to formulate theories; c) theories from general to even more specific ones; d) languages, speeches, logical symbols, concepts and expressive terms.
The enthusiasm for the depth and rigor of scientific discourse should not turn into praise. In a simplistic way we can come to be regarded as 'non-issues' all the questions that science excludes 'in principle' to be able to deal with. That statement is a sign of a bleak 'poverty of imagination'. It is not with the removal that the problems are eliminated, even though, when we have recognized them, we have to admit to be unable to solve them in an absolute manner.
Qualifying knowledge is not only to give answers but also and above all to present problems. The unavoidable need to address the problems arising from the relationship between theory and practice is expressed by the term ‘orthodoxy-orthopraxy’, which indicates the inseparable link between theoretical research and practical proposals. The growing problematic nature of the activity, methodology and scientific knowledge is a key challenge for the intelligence and reason of man.
Restricted and limited rationality are not sufficient to meet the needs of the spirit and of human culture. If we become aware of those, and how humanly important are the problems that remain inaccessible to the logic of science, we my say that we are still in the dark . The finding may seem disappointing. But it is objective, necessary and can be healthy. In fact, it is disappointing only for ideologists and ‘scientists’, not for real researchers.
If we want to go towards a new humanism, understood not so much in the traditional literary sense, but rather as a unified and balanced vision of man, it is necessary that those who deepen the naturalistic studies and those who meditate on its trans-natural dimension do not lose reciprocal contacts, keep the conversation alive and dialoguing and by dialoguing do not behave as advocates of a thesis, but as ‘searching’ men.
The quantitative versus qualitative debate has taken place within our RG research team and significant steps towards a reconciliation of initial differences is taking place. As a means of introduction of the following considerations, I recall to have noticed quite a synthetic but effective message on our task of research designers:
As a starting observation I’d like to point out that the point at which the data analysis begins and ends depends on the type of data collected, which in turn depends on the sample size, which in turn depends on the research design, which in turn depends on the purpose.
Nevertheless, far more insidious discrimination remains. Systematic review within the topic of methodology exhibits all the characteristics of "institutionalised quantitativism" in that criteria for a "good" review are almost entirely determined by the quantitative methods promoted. Nobody who understands qualitative research would insist that its primary studies demonstrate alien concepts such as "sample size" or "statistical power". Yet comparably fundamental incongruities persist with regard to qualitative syntheses. Why should systematic reviewers of qualitative research pursue a "gold standard" comprehensive literature search when concepts such as "data saturation" have an established pedigree? Why shouldn't they apply systematic, explicit and reproducible principles of thematic or concept analysis to create syntheses that advance our understanding of qualitative issues and highlight research gaps?
Due significance should be given to the methodological options at our disposal reckoned to be more appropriate in the different circumstances dependent on the nature of the study. Accordingly, we should make choices of methods that are both philosophically defensible and, at the same time, practicable and responsive. For instance, taking the case of researchers becoming aware that the purposes of their study often involve both quantitative and qualitative aspects, it follows that it will be more appropriate to explore the opportunity of developing mixed method research designs that may be better associated with their investigation purposes.
Then, following the ideal thread of earlier observations, I choose to concentrate on the ‘mixed methods’ methodological approach employed as a research configuration in social sciences at large especially in cases of integration or connections of quantitative and qualitative data.
On the first approach I observe that an increasing development of mixed methods research is commonly accepted as a distinctive feature of contemporary research designs to profit from the inclusion of both quantitative and qualitative sources of information, mostly when generalization of results and feedback evaluations are the purposes pursued. In fact, as it has been remarked, the insertion of qualitative data can help researchers to enlighten relationships emerging from quantitative data. Similarly, the inclusion of quantitative data can help in compensating for the fact that qualitative data normally cannot be generalized.
According to the prevailing literature on mixed research design , combining quantitative and qualitative analyses has been advocated when the process presents evident complementary strengths as in what Denzin (1978) dubbed triangulation of different data source, i.e. the process of testing the consistency of findings obtained through different instruments for the study of the same occurrence. Specifically, the combination of the two approaches seems useful when:
1) results from qualitative interviews can help to identify unobserved heterogeneity in quantitative data as well previously unknown explaining variables and misspecified models;
2) results from the qualitative part of mixed methods design can help to understand previously incomprehensible statistical findings;
3) qualitative research can help to discover quality problems of quantitative measurement instruments;
and 4) quantitative research can be used to examine the scope of results from a qualitative study and support the transfer of such findings to other domains” (Kelle,2005).
For its characteristics, mixed method research has been labeled: the third major research paradigm and gained the legitimacy of being a stand-alone research design. However, besides its strengths, controversial issues still hinder its potential and some of the details remain to be worked out by research methodologists (e.g., problems of paradigm mixing, how to qualitatively analyze quantitative data, how to interpret conflicting results). Then, a word of caution seems appropriate to be addressed to researchers (and better to research teams) who plan to design inclusive, complementary methods that are capable to embrace diverse perspectives, data and values within and across studies.
References:
Denzin, N. K., “Triangulation”. In N. K. Denzin (ed.), The research act: An introduction to sociological methods, 1978, McGraw-Hill, New York
U. Kelle, Mixed Methods as a Means to Overcome Methodological Limitations of Qualitative and Quantitative Research, Workshop on mixed-methods held on October 26-27th 2005 at the University of Manchester.
Hi Mr Chan,
I think the 11 interviewees (for your qualitative analysis) should come from the 300 samples (for your quantitative analysis) you had tested since you are adopting the mixed methods sequential explanatory study. However, based on the challenges you had mentioned that you couldn't obtain the interview sessions with those that had responded to your survey, perhaps you might want to try either one of the approaches:
1) Perform Anonymous Online Interview - you didn't mention how you'd conducted the survey either via online or paper & pencil, but you can develop an online interview (similar like online survey except you ask open ended questions, so that informants can key in their answers in long text format) & email to the same / smaller no. of respondents to solicit their answers toward your interview questions. You can also distribute the hardcopy interview sheet to them as well if online method is not in use. The rationale is if they can fill up your survey anonymously, they might provide the "interview" answers anonymously. Also you don't document their views, they are document their own views. Moreover, they don't fear losing their jobs as the input / answers provided are anonymous just like the first part survey.
2) Ask the 11 Interviewees Respond to Your Survey - For those 11 interviewees' answers you had collected in which they were not originally from the 300 samples, can you ask them to fill up your survey questionnaires so that they are part of your sample now i.e. total sample is 311.
For either one approach from the above, you need to pen down in your thesis the limitation of each approach. For example, Approach 1 is not genuine face to face interview in which gesture, tones etc were not observed etc. (try to review some literature that there are precedents of such "interview" method being taken place & accepted). For approach 2, there is limitation of not getting random sample for the 11 respondents in which they are based on convenient sampling etc. (try to do some literature review to find evidence to justify that this kind of convenient sampling is acceptable in abnormal situation)
Sometimes the examiners can understand this kind of real challenges that the researchers are facing and accept the limitations that the researcher had documented. All the best.
Regards,
Fung
- Badges
- Science topic