Times without number, I have read in many studies where these tests were expected to be chosen especially when the sample size not so many but thought to be considered for non-parametric tests, however, the choice would be seen considered for the common parametric tests. Again, in a study with many groups, each of the groups having sample size less than 40 each but altogether are more than 100 participants/models, what is statistical choice advisable?.
.
when you perform parametric statistical tests, you must ensure that your data fulfill the hypothesis of the test, that is that the data distribution follows some predefined law (for anova, that the distribution of the variable is normal within groups with the same variance, for instance)
if you cannot ensure this (the data have a distribution definitely different from expected by the test hypothesis), then you will fall back to non parametric tests which often have less power than their parametric equivalent
.
the size of the sample does not enter in the above, in first approximation at least !
now, for small samples you may be unsure of your data fulfilling the hypothesis of the parametric test (for instance, you have too little power for your gaussianity test ; of course you cannot reject the hypothesis that the data is gaussian ... but you are still pretty unsure !), in which case you may prefer to fall back to the non parametric tests ... but with the danger of having too little power (small sample sizes and non parametric tests do not mix very well !)
.
I think that the first consideration in deciding what statistical test to use is to decide what hypothesis you want to test.
For example, t-test, Mann-Whitney, a median test, and 2-sample Kolmogorov–Smirnov test all test if something about two samples is similar, but each tests a different thing.
For example, imagine we have the household income of two towns, and this income is strongly right skewed, which is typical *. Even if a t-test is appropriate, we may not be very interested in measuring a difference in means, since means may not be the most meaningful measure for this kind of skewed data. Perhaps we would rather compare the medians, since medians are what are usually used for income data. For this we might use Mood's median test, or a permutation test for medians, or quantile regression. But maybe we want to open a high-end store and want to know if there is a difference in the top 10% of income. Then we would want to use a test for the 90th percentile (again, a permutation test or quantile regression). In other cases, the hypotheses tested by Mann-Whitney or Kolmogorov–Smirnov may be what we want.
I try to discourage the thinking that e.g. a t-test is the default test, unless our data don't meet assumptions, then use a traditional non-parametric test. One reason is that I discussed above: these tests test different hypotheses, and you should start with a test that tests the hypothesis you want to test. But also because there are other robust methods, other than traditional nonparametric tests, that can handle most situations where data deviate from classic parametric assumption. Furthermore, there are other parametric approaches that may be more appropriate for the data in question. This might include generalized linear models like Poisson or gamma regression, and things like ordinal regression and quantile regression, which are sometimes called "semi-parametric."
The drawback here, is that, you'd have to be familiar with a bunch of approaches and tests, and when they are appropriate. But if we can empower researchers and students to have a "toolbox" of approaches, they are likely to find the approach that respects their data and answers the questions they are trying to ask.
___________
* Few people are relatively wealthy and most people are relatively not wealthy.
- Badges
- Science topic
- Similar topics
- Mathematics
- Statistics
- Sampling Design
- Sample Size