Determining the optimal sample size for a research study depends on various factors including the purpose of the study, the research design, the type of data you are collecting, and the statistical analysis methods you plan to use. In your case, it seems like you are conducting a pilot study to explore the correlation between physiological measures, survey responses, and group discussion context. Running two groups of 8 participants each could be a good starting point for the pilot study, but keep in mind that the sample size you choose will impact the power to detect differences or correlations and may affect the generalizability of your findings.

Hi Sophie,

Your analysis plan (for the full study) will indicate your main comparison of interest, which in turn will identify your within and between sources of variance. Your proposed pilot will have K(Groups) = 2, N(Types) = 4, and n(Subjects) = 16. Samples of 2 and 4 will not yield reliable statistical estimates of Group and Type things you may need to adjust for in your final analysis such as intraclass correlation. Perhaps the best you can do is use an online calculator to calculate the confidence interval you would observe across a range of plausible r for n = 16 and make a reasoned argument that the precision of your estimate will be sufficient (assuming ICC to be negligible).

For your ability to identify problems in your design (perhaps a stronger use of pilot studies than estimating effect sizes) there are papers such as this: Viechtbauer W, Smits L, Kotz D, Budé L, Spigt M, Serroyen J, Crutzen R. A simple formula for the calculation of sample size in pilot studies. J Clin Epidemiol. 2015 Nov;68(11):1375-9. doi: 10.1016/j.jclinepi.2015.04.014. Epub 2015 Jun 6. PMID: 26146089.

First, you need to determine the purpose of your study and the research questions you want to answer. This will help you decide on the appropriate sample size for both the quantitative and qualitative components of your study. Second, you need to consider the type of mixed methods design you will be using. For example, a convergent parallel design involves collecting and analyzing both quantitative and qualitative data separately and then comparing the results to see if they converge or diverge. The sample size for each component of the study will depend on the specific design you choose.Third, you need to consider the resources available to you, such as time, budget, and access to participants. These factors can impact the feasibility of recruiting and retaining participants, which can affect the sample size. Finally, it is recommended to conduct a pilot study to test the feasibility of your research design and to refine your sample size estimation.

Depending on the purposes for your pilot study, two groups of eight could well be sufficient.

But when you do your ultimate quantitative analysis, you will need to take into account the non-independence of your nested observations. I suggest you look into multi-level analysis to gain an understanding of this issue and how to analyze data that take this form.

Dear David, Hameed, Mark and Ajit

Thank you so much for taking the time to share your thoughts. Much appreciated & very useful both for this stage and the full study.

With kind regards,

Sophie

Hi Sophie,

You can additionally consider introducing another method to compensate a small sample. For example, interactions before and after the recorded online discussion could be examined. Or participants' attitudes toward the course/ course instructor, etc. Alternatively, you could look at discussion posts and calculate/ analyse them using content analysis (ie quantifying the qual data). I'm not sure this particular suggestion answers your question but the main idea is to a) have data triangulation to ensure the study is robust, b) consider different analytical approaches to see how qual and quan data can be integrated at the mixed methods stage. Needless to say that any study has limitations and delimitations and this could be a starting point of assessing your sample size in the pilot study.

All the best,

Assel