Hello Mark,

My main project was grounded theory and I stopped interviewing when I reached data saturation. Although this does not answer your question about "gut" feelings, I did feel that I was ready for the next stage without feeling concern about this. However, I did have really good support from my supervisors, throughout my whole study, including my moving on at this point. Perhaps - more generally - the consensus of more than one person is valuable.

This is a good question and it will be interesting to read the responses of others.

Very best wishes, 

Mary

Many thanks, Mary. I put the question in a very general, very open-ended way, purposefully with no discussion of my own research, to try to elicit maximum discussion, from those of varied backgrounds. Of course, as you imply, what you feel able to defend will always be an answer, but I will say that I think we do have preconceptions over that, however personal or otherwise. I will mention my research to the extent of saying I was hugely surprised how soon solid themes appeared to emerge. In fact, I'll go further (though so far this is reflective, no direct questions asked of participants) and say that I think one major theme, held with a vengeance in some cases, relates with local conditions in the particular part of Australia where I'm researching.

Hi Mark and Mary,

To me, the gut feeling needs to come with the chosen sampling technique and the anticipated number of participants to achieve what it is that you want to achieve. If done correctly, then repetition (rather than saturation) should occur naturally. Mary - Grounded Theory is designed to achieve theoretical saturation - but this is not the case with most other qualitative approaches. Mark - the intention of qualitative research is not to generalise - although contexts can be generalised i.e. if you are researching bereavement then it can be 'assumed' that most people will view it as a negative experience. 

In terms of 'how many is too few' depends on a number of different factors i.e. the available sample population, the sampling technique employed, the methodology and method adopted etc. The attached chapter on sampling and data collection may assist. It's nursing -related - but the principles are generic. 

These links may help you

https://www.quora.com/Why-is-30-considered-the-minimum-sample-size-in-some-forms-of-statistical-analysis

http://www.dummies.com/education/math/statistics/how-to-determine-the-minimum-size-needed-for-a-statistical-sample/

https://www.quora.com/What-is-the-smallest-sample-size-needed-to-achieve-statistical-significance

https://stats.stackexchange.com/questions/37993/is-there-a-minimum-sample-size-required-for-the-t-test-to-be-valid

https://www.researchgate.net/post/What_is_the_minimum_sample_size_regardless_of_total_number_of_population_that_can_be_considered_as_reliable_for_performing_advanced_statistical_tests1

Dear Mark

The size of the qualitative samples can not be established a priori. I work with interpretative thematic analysis and just as Mary I use the point of saturation as a guide, that is to say when new categotions no longer arise in the inductive analysis of the data. From my experience in the area of consumer behavior most qualitative studies find this saturation point between 15 to 25 respondents.

Greetings,

Jorge

Excellent Jorge. I do believe that one can develop an instinct in these things - meaning really in one's particular area of study. Whether one should, or whether that starts to pre-determine is a whole other discussion! Now, quite rightly, you've been very clear in talking about your field being consumer behaviour. My "gut" felt your figures quite high. Now trying to rationalise that, I went to medical enquiry, where sometimes the number of participants is very low, sometimes there aren't even that many patients in a category. My own field, producer behaviour, I feel different again. I should add that I'm talking about very small-scale artisanal food producers.

Dear Mark

Very interesting your answer. I would add another guideline that applies to both qualitative and quantitative sampling. The more homogeneous your study population is, the smaller the sample size needed. In the case of qualitative sampling, a homogeneous population of study produces to reach the point of saturation more quickly.

Jorge

Absolutely to everything you said, Jorge. Absolutely to the point of hilarity - it's as though my little cadre of artisanal food producers was built on a production line (boy they'd hate to hear that, not only for the associations, but they like to think of themselves as rugged, different, individual and free ..... all of them!!). They, of course, have location in Western Australia in common, they have aspects of Western Australia's recent history in common, they have being artisanal food makers in common, but what they actually produce is very varied, and often comes out of very, very personal experience (having helped to cure a loved one has been a remarkably common starting point, etc).

Dear Mark

Thanks for the nice talk and good luck in your research.

Greentings,

Jorge

the more parties involved, the better the data is generated. this will affect the quality of data obtained.

I agree with Jorge Cruz-Cárdenas that the homogeneity of the underlying population is a major Issue, which I think is too often ignored in qualitative research. Thus, if there is rather little variation in what participants have to say, then this will produce saturation very quickly. In contrast, if there are a number of different points of view, it will take more interviews to thoroughly capture that kind of variability.

So, my gut feeling is that there is no one gut number.

I would add that if the first few interviews reveal a lot of variability, then it might be wise to redefine the research question or the selection of participants, to ensure a manageable number of interviews. In particular, trying to analyze a large number (whatever that is) of interviews can be quite a burden.

Mark, as you point out - the reason you are reaching 'saturation' so quickly is because your sample is too narrow. Is there a measure of 'saturation'-does this mean the same or similar responses to some themes or all themes of qualitative discussion? Many qualitative studies are limited by budget and time or both and the 'gut' applies when too few is assessed as being just enough to reach your conclusions. Not ideal but then that may be the reality.

I take your point, Glenn. The simple answer is extreme limitations of budget and time, when based in Thailand and self-funded, but researching in Australia. Equally, though, whilst I could widen, particularly spatially, that was never my aim. It was always to consider very small scale artisanal food producers in the Perth/Fremantle area of Western Australia. That isn't a huge number of potential participants! But, seemingly, a rather cohesive cohort. Now, interestingly, that partly results out of fully-planned reasons. I know the Perth/Fremantle area quite well. I "carry" the background. Some would say fateful to qualitative study. I took the opposite view, whilst not bringing background into my actual questions. They were the same as would be the case anywhere. But when participants spat out "Freedom" as a motivation, I readily felt the background of the collapse of minerals and mining coming into play, if only in terms of the vehemence. In fact, participants liked to talk through their personal histories, including redundancy, non-renewal of contracts, etc, without prompting, so those are available in audio file or transcript.

I concur with David’s view that “there is no one gut number”. Nevertheless, there is no one-size-fits-all method/numbers for all studies in reaching data saturation, which should relate to the depth of data rather than the numbers themselves (Burmeister and Aitken, 2012; Fusch and Ness, 2015). Another question/discussions as to an ideal sample size for qualitative research may be able to provide additional/interesting views, vide: https://www.researchgate.net/post/Whats_is_ideal_sample_size_in_qualitative_research/2 

• Burmeister, E. and Aitken, L. M. (2012). Sample size: How many is enough? Australian Critical Care, 25, 4, pp. 271-274.
• Fusch, P. I. and Ness, L. R. (2015). Are We There Yet? Data Saturation in Qualitative Research. The Qualitative Report, 20, 9, pp. 1408-1416. 

Mark, if your audience is small perhaps you need to broaden your themes. I assume you started with FGD and then individual interviews based on the key themes?

No. The approach was quite the opposite. The starting point was singularly "small-scale artisanal food producers in the Perth/Fremantle area".  Well, of course. that's being a bit untruthful, since the idea was to discover their motivations, but the themes emerged from the discussion. I made no attempt to even think through what might motivate the participants. I largely let them move conversationally as they wished to that tune of what motivates them, pushing the conversation back into relevance where necessary, as it can get very anecdotal. So their themes, albeit by my consideration, arose out of the interviews. There was no focus group. Absolutely zero ability to create that at distance but what was possible was fitting in odd meetings in odd places and at odd times with individuals. These are rugged, individualist farmers and ex-minerals workers. Perhaps the whole point is that they're avoiding a life of conformity to administrative paradigms. Now you could say that my sample set of interviews constitute the focus group and I could now move on to individual interviews off the back of the themes, but the problem then becomes a numbers game. I've already interviewed a significant percentage of the very small number of people who fit the stated profile, and, of course, I've accessed the most accessible. In fact accessibility is an interesting point to the extent that I think there will be people out there who I've not heard of and never will. Looking at it from my perspective I can see a case for a low profile, even aside from the taxman, in that a very strong position to be in is direct supplying to one or two restaurants with absolutely nobody else aware, so able to steal the contracts. Western Australia is awfully small town. Walls have ears. It may be that you just loose kudos by it being heard that you supply ...... Oops, nearly put in a specific name then, of a company notorious for paying low!

Mark, what is the small number of people you have to interview?

The total available cohort, in fact being slightly elastic with my definitions, is only 40, maybe 50 artisanal food  producers. Where I am being definitionally strict is around food only. In Western Australia the picture would be very different if one looked at wine. The picture is as per better-known South Australia. The other thing that would change the picture is looking at producers of farm produce, farmers simply growing vegetables or meat stock, sold with no on-processing. Finally, I'm limiting to Perth/Fremantle, not the whole of Western Australia, to "square" with other research I've done in that area.

I am therefore not sure why number is the issue-so best to review your style of qualitative assessment. Good luck.

Enter your answer

No number of instances can confirm a generalisation - this, the problem of induction is one that has puzzled scientists and philosophers of science since at least Hume highlighted it. How do we know that a bark will always be followed by a dog or that night will follow day? How can  the inferences of uniformity and predicatability which form the ' the cement of the universe' as Mackie termed it be justified? 

I think the problem of generalisation is insoluble as many have acknowledged despite looking for solutions in probability, causation, verifiability, falsifiability. Popper felt the problem of induction and scientific discovery  was best solved by ditching it and focusing instead on deductive inferences and justification.

This is all fascinating and complex stuff which has no easy answers.

Whyeda, it is indeed complex, especially when looking at qualitative studies. I'm not that well-read on qualitative, but I've not seen anything that particularly convinces me in this matter, and possibly the same applies to anybody. There is no agreement on generalisability. Statistical generalisation in quantitative work, on the other hand, notwithstanding that there are critiques, is widely accepted. Let's put this simply, as is my way. You walk into a quantitative defence and say I put out this questionnaire, I had over four-hundred useable returns ..... . You know you are on safe ground in that matter. With qualitative there isn't that known "safe ground". And this is where I go back, notwithstanding all the discussion to ideas of "gut feelings". There are exceptions, such as a very small patient cohort, but I think if you walked into a general defence and said I interviewed three people, it just wouldn't feel right to committee - you'd have a huge amount of justifying to do. Increase that figure to something that accords with gut feelings (however ill-informed), which again I repeat I'm sure people have, and you won't have the same level of problems, The psychology could be something, say, like double-figures matter, so your arguments around saturation would have difficulties for the three, even if you not only felt saturation was achieved at that level, but that, in your mind, you could justify that. belief.

Mark, I think part of the difference you are describing arises from the difference between the purposes of qualitative versus quantitative research. In quantitative research, when one tests a hypothesis, one want to know the population to which one can generalize those results. Hence, one relies on a well-developed literature on statistical sampling. In contrast, qualitative research is more likely to generate rather than test hypotheses. Hence, one relies on concepts like saturation to assure the thoroughness of one's data collection.

A different way to look at this is to ask what implications you would like to draw from your research conclusions. Given the goals of quantitative research, generalizability is a highly relevant issue. For qualitative research, I prefer an emphasis on "transferability" (Lincoln & Guba, 1985), which asks about the other settings were the current results might have relevance. This also amounts to generating a hypothesis: If my results are reasonable, then they should also apply to...

Many thanks, David. I think that exactly hits the nail on the head, and I myself have come to understand that tranferability is rather different from the more deterministic statistical notions within quantitative research. The trouble is that we aren't just talking difference, but greater opacity in the case of transferability. I think my own studies perfectly prove your point. As said, I asked my small-scale artisanal food producers about their motivations with no pre-conceptions on my part. To take one theme that emerged, sometimes , actually mostly, howling, sometimes less so, but coming out of every interview conducted -  Freedom - it seems to me that one could generate hypotheses out of that. In fact in my mind the only question would be the applicability of this to small-scale artisanal food producers as opposed to small businesses in general, or is that start-ups in general, or, particularly start-ups in a particular place or at a particular time  I mention the last not simply as a general intellectual point, but from my untested belief that the vehemence with which freedom has been mentioned is connected to the collapse of the mining and minerals industry in Western Australia. That was a route to arrival in small business by over half my interviewees so far. It is very interesting that I should have detailed knowledge of employment histories (though, in free conversation, all interviewees wished to provide those) - I think both that they figure large, and may well be associated with 101 different categories of recent start-up in Western Australia. Then, there's the difference between start-ups and longer-established businesses (which actually began to show through even in my small number of interviews to date). Next, there's the Western Australian character, which is arguably rather "frontiersman". One interviewee seemed constantly quizzical about how he'd ever ended up in an office in the first place! ..........

The goals of qualitative study are important and the number of respondents would be sufficient if these goals are achieved. However, as a thumb rule the respondents should be taken as minimum five to avoid any biases.

This will depend upon case to case basis also and profile of the respondent, whether he is well versed with the problem and able to answer correctly or not. However, i will go with Prof. Sushil that minimum 5 respondents needs to be there to avoid biasness.

I allow myself to react in another direction. Qualitative research starts from the premise of seeking the understanding of a problem, from particularities. Its nature in itself does not imply seeking generalizations. However, there are texts that aim to develop methodologies to make inferences from qualitative research (The design of social research: scientific inference in qualitative studies - authors: Gary King, Robert O. Keohane, Sidney Verba).

When referring to case studies, it is because of the research problem, from which we started. We identify cases that allow us to better understand what is happening, and to answer the research question raised. Therefore it can go from 1 case to many cases. The main thing is in the demonstration that the selected case (s) have the characteristics that will allow to answer the research question.

Actually 20 is a number that I have been faced with in many qualitative research papers, with the recommendation of observing a "plateau: saturation" in terms of the information collected. 

based on my readings, in a quali research, if you have homogeneous participants, the least number of your participants is 10-15

That seems to be my conclusion, but from reading, not necessarily from practice. I have adopted that model and no doubt will also do so in the future, meaning say simply fifteen interviews. Saturation is interesting though, particularly when working across locations. In one study I was comparing Australian and Thai participants. Having completed three interviews in Australia I felt that's it, saturation, but continued to clock up to 15. Sure enough there was no information gain through the extra 12 interviews. In Thailand, I interviewed 8 or 10 people and still concluded "this is all over the place" (meaning in terms of themes[ except the very broadest] and categories). I continued on to 15 interviews and nothing was resolved, no saturation, new categories still emergent. I would speculate that I was dealing with the difference between the Australian and Thai persona and that I could could have undertaken 101 interviews in Thailand and not captured the fullest range of input.

From reading, I would agree. I have indeed used a straightforward numeric proposition of 15 on occasion for interviews. I have no doubt I will do so again. But saturation is interesting. I was once interviewing and comparing Thai and Australian participants. I interviewed 3 Australian participants and said "job done", saturation reached. And sure enough, I interviewed 12 more and gleaned no additional information to contribute to themes or categories. With the Thais, I reached 8 or 10 interviews and said "this is all over the place". I continued through to 15 interviews and still felt I'd not exhausted all possibilities for categories. The Thai answers were highly random. I believe I was dealing with a difference in persona between Australians and Thais.

In my own opinion, l agree with some of the recommendation of samples or participant that says 10 -15 because of the duration of interviews, numbers of memos, audio /video recording and locations of the participant from the researcher.