The population of this research study are students from a specific school, and 6 strata was identified. The other 3 strata met the calculated sample size required, however the other 3 unfortunately did not despite sending out the surveys again and again. Additionally, a majority of the students were minors, and there were some whose parents did not give consent for the survey.
With this, the 3 strata were not able to meet the calculated sample size. Should we proceed with statistical analyses (with 3 reaching the calculated size, and the other 3 not) and write this as a limitation? or what other actions should we take instead?
By "calculated sample size required," I assume you refer to a power analysis done before data collection to calculate the minimum sample size required for obtaining statistical significance with a given probability of Type 1 error and a given minimum effect size. If so, then yes, you can proceed with the analysis, while acknowledging that the shortfall in desired sample size will necessarily increase the probability of Type 2 error.
You should also up-weight the respondents in low-response strata to compensate for under-representing the subpopulations in those strata. Unless your sampling involved strictly proportionate stratification, you would have to do some weighting even if all strata had the same response rate.
Of greater concern is whether the low response rate in some strata reflects a non-random bias in non-response within those strata. If you can obtain subpopulation figures for each stratum from other sources on key demographics (like age, gender, etc.), you can adjust (to some degree) for such bias by doing post-stratification weighting within each stratum, to make your sample demographic marginal percentages match the known subpopulation values. If you only have demographics for the whole population, you could do post-stratification weighting for the sample as a whole rather than within each stratum, although the within-stratum approach would be better.
By the way, with the population being students from a single school, unless that school has a huge enrollment, both your power analysis and your statistical inference should include the Finite Population Correction (as you may already know).
Try to assess whether the strata with sufficient sample sizes are representative of the entire population or if they might introduce bias, and at the same time, Contemplate the characteristics of the strata that did not meet the sample size. Then proceed with the statistical analysis for the strata that met the sample size requirements to see your result. After that, conduct a sensitivity analysis to see if the outcomes have similar results. Subsequently, try to disclose your limitations and clarify the future study.
- Badges
- Science topic
- Similar topics
- Psychology
- Mental Processes
- Learning