I am running an experiment to compare the difference between the control and intervention groups with 5 and 4 sample sizes. From the literature, the smallest sample size that an independent sample t-test can be used is 4. However, one of the major assumptions of the t-test is that the data from each group must be normally distributed. Normality assumption cannot be established on a small sample due to inadequate estimation of the dispersion of the data.
Can I continue using the independent t-test or consider using a non-parametric test. I have run 70% of the experiment data before the normality assumption comes to mind.
The assumption should be sensible and resonable. Judging if this is the case requires to understand the characteristic of the variable you are measuring. If you don't understand your variable, you possibly should not test hypotheses at all, as you will not be able to interpret the tested hypotheses anyway. If you have some justifyable opinion that the variable should have an approximate normal distribution but you are not sure if this is sufficiently correct, the assumtion is quite save or conservative. The problem may only be that the data remin unconclusive, what is your personal problem but not a scientific one.
That brings me to using non-parametric tests: usually, these tests are about a different hypothesis, and again you should be clear about which hypothesis you actually want to test. It is not useful to test some weird or misunderstood hypothesis just because one knows how to test it. There should be a scientific rationale behind your choice (a feasability argument is not a scientific rationale). Despite this, your tiny sample size prohibits non-parametric analyses anyway. There is not enough information in the data to reliably estimate the sampling distribution, and non-pareametric tests will have almost no power (again: power to test a hypothesis that may not be the one you actually like to test!).
Yusuf Olatunji Bello , Are you considering the variances of the groups also (homoscedasticity)? Or perhaps you are using a t-test that is appropriate for groups with unequal variances ? ...
It sounds like you have many tests to run, based on the comment about 70% of the data being run. Are your data such that you can pool them in any way to examine the distribution of observations ?
There are several different tests that could be run to compare two independent groups depending on what you want to test: mean, median, ranks, and so on. You might re-consider which hypothesis is most useful for your data. It may be that considering the unknown distribution of the underlying populations you are examining, that a hypothesis about medians or ranks may be more of interest for you. You might also consider using a permutation test.
Hello Yusuf,
Regardless of how you proceed, the biggest hurdle you face with respect to determining whether the two samples differ by a sufficient amount to infer that the same is true for their respective populations is the very low statistical power (mentioned by Jochen Wilhelm).
If normality of model residuals is the key concern for you, you can always apply an exact test. The computational demand with n1 = 5 and n2 = 4 is very low (a total of 126 comparisons). However, power is still low unless the magnitude of the difference is very larger.
Finally, as a trivial case, one can evaluate an independent t-test with as few as two cases per group. Doing so is not a wise choice, however, due to the exceedingly low power for testing the null hypothesis.
Good luck with your work.
- Badges
- Science method
- Similar topics
- Mathematics
- Statistics
- Statistical Modeling
- Analyses of Variance
- ANOVA