Hi all,
If I wish to compare data from two groups, one of which has data that is normally distributed and the other which has data that is not normally distributed should a parametric or non-parametric test be used?
Also, I am confused with regards to the central limit theorem and its implications on data. Based on the CLT, is it true that all data collected from a population is normally distributed? If so, then why is there a need to use non-parametric tests, or check for normality in the first place?
Thank you for your answers in advance.
What test you should use depends on what kind of hypothesis you want to test in what kind of model. If no test is available and your samples size is not too small you can do bootstrapping.
You say you want to compare two groups... what is the observed variable? If this is a nominal variable, you can compare frequency distributions of the outcomes; if it is a metric variable, you can compare the cdfs, or assume some distributional model and compare model parameters or model statistic.
The CLT has no impact on data whatsoever! The CLT has an impact on statistics that are calculated from data. It says that the shape of the probability distribution of a sample statistic that is based on a sum of values is approximated the normal dirstribution model, and that this approximation gets better with increasing sample size. For instance, the sampling distribution of the mean difference will be approximated by the normal normal probability model, almost irrespectively* of the distribution model assumed for the data, and this approximation will be better for larger samples sizes. Thus, having a reasonably large sample size allows you to use a test on the mean difference that assumes that the mean difference is a normally distributed random variable (-> t-test), even when the assumption of a normal distribution of the data (within the groups) is not quite reasonable.
----
* A counter-example is when you should assume that the data has a Cauchy distribution (e.g. the data are positions of a laser-spotlights of on a wall from a laser-pointer at some distance in front of that wall pointing in some random direction). This distribution has no mean, and no matter how big your sample size gets, you will never get a satisfactory approximation of the distribution of the sample mean to the normal distribution.
Dear Nathanael,
One small point to add to Jochen's excellent answer: some parametric tests still give reliable results even when their assumptions (such as normality) have been violated. This is known as robust use. The conditions for such use have been established using simulations and vary from test to test. This approach complements the CLT issue which is more relevant to large sample sizes.
Hello Nathanael,
These are very common issues when comparing 2 populations.
If your sample size is small, go with non-parametric tests like Mann-Whitney. The advantage of this test is, even if both the distributions are normal, the results don't get affected.
If your sample size is large, you can generate 2 distributions of any comparison statistic (better to take Mann-Whitney U Value), one with labels permuted, another with bootstrap sampling. Compare these 2 distributions using Z-score test. Check the p value obtained.
Also, you can use Kolmogorov–Smirnov test.
I hope this helps.