Dear Nur,

before you can determine the appropriate test, you have to find out the distribution type. Because you have a small sample size, I suggest the Shapiro-Wilk test. After performing the test, we can choose the right test. In this calculation, there are no correlations, but comparisons between 1) two or 2) more than 2 groups. Thus, you have to chosse following test:

For normally distribution:

1) t-test

2) ANOVA

For non-normally distribution:

1) Mann-Whitney-U-Test

2) Kruskal-Wallis-test

Best regards

Tanja

Hi Tanja,

Thanks for your answer. How do I perform a t-test if the number of students for each course is different? Do I average the scores?

Regards

Nur

Hi Nur,

this is not important for the analysis - you can compare the both groups.

Regards

Tanja

I follow Tanja Kottmann answer

I recommend non-parametric tests, Tanja proposed. This is because your sample groups are not equal and your dependent variable is actually non-continuous, but rather ordinal. For the correlation, Spearman's rho should be the best.

Dear all,

The Spearman´s rho is a completion after comparison of the groups. I recommend to calculate both and so you have much information about differences of groups and trends during the time.

Best regards,

Tanja

Of course, you may test the differences between the groups and the correlation to have a full picture. Remember only to specify the hypotheses a priori in order to not compromise the meaning of p-values. If you test multiple hypotheses, please adjust the results for multiple testing (Benjamini Hochberg or Bonferroni procedure).

I think these different test are very hypothetic and therefore, I prefer more commun tests.

@ Tanja: 1) which tests are hypothetic in your opinion?, 2) why using "more common" tests, if their application is not justifiable, e.g. if assumptions are not met? You only distinguish between "normally distributed" and "not-normally distributed", but these are NOT the only assumptions that have to be met to conduct a t-test or a ANOVA!

1) the dependent variable has to be metric, and test scores may be metric, but could also be only ordinal (this depends now on your judgement)

2) for parameter estimation and inferential statistics, the data should be (and the sampling distribution has to be) normally distributed. I.e. you should carefully exam your data and look for outliers (scatterplot, QQ-Plots, tests for normality, test the residuals on normality etc). You cant rely on the central limit theorem, since your sample size is very(!) small, and even if it would be large enough, this does not guarantee that it works (c.f. Wilcox - Fundamentals of Modern Statistical Methods). Use bootstrap techniques or robust estimators like Yuen estimator.

3) Homogeneity of variance: variances in both samples have to be equal, if not, use the Welch correction or use onther robust estimator (see Wilcox on that topic) or switch to non-parametric tests.

I also support Michael answer that it is mandatory to specify your hypotheses a priori and to control type 1 error using adjustments for the p-value, like Bonferroni or Benjamini-Hochberg, if you conduct more than one test. I think it is also mandatory to report effect sizes, since null hypotheses testing is flawed (especially if done wrong).

Hello Rainer,

the most important key point to get a reliable statical analysis is to have the right study design, good questions and end points, correct tests and to be able to handle the software. The question from Nur is very basically.

In my daily practice, I see numerous studies with wrong questions, wrong endpoints, with bias and with totally wrong results and conclusions. Many of them try to take very sophisticated test and most of them fail.

Thus, I think, if somebody ask a very basically question it is helpful to explain:

1. It is important for metric parameters to perform a test of normality

2. to choose the right test and to distinguish between f.e. ANOVA and Kruskal-Wallis.

The test I mentioned are quite established and so, I again recommend them for first steps in statistical analyses. They are reliable! Thus, your explanations are not in contrast to my recommendation, but I think my way to start the analyses are more practical.

Dear Tanja,

you may be right that these tests are more practical, well established and sometimes even somehow robust, but nonetheless, using a test because everyone is using it, despite the fact that it should not be used (e.g. due to violations of assumptions) also leads to " bias and with totally wrong results and conclusions".

Why do you stick to this silly normality assumption in particular but not the other ones? Why is this more important than other ones? Even small violations of normality (e.g. heavy tailed distributions, which you cant simply detect with typical checks) can lead to massive loss of power. A simple switch to nonparametric test is not an appropriate solution. And no, therefore these tests are not reliable, quite the contrary! And you cant predict if this will lead to increase in type 1 or type 2 error.

We should see it the other way round: because we can assume that most of the time these precious assumptions are violated, we should consider using robust methods in the first place and establish them even more instead of sticking to old habits. As you pointed out, we should use reliable (!) instruments and not the old ones per se. For example, a recent article showed, that it is more advisable to always use the Welch test instead of Student's t-test

Delacre, M., Lakens, D., & Leys, C. (2017). Why Psychologists Should by Default Use Welch’s t-test Instead of Student’s t-test. International Review of Social Psychology, 30(1).

Rand Wilcox dedicated whole books on robust tests using metric information instead of using nonparametric tests.

Hello Rainer,

I can just repeat my last answer. You instruct me in details which are not appropriate to answer Nur questions. By the way - I myself use test like the Welch test every day - but this it not interesting for the question. In summary, we both found another ways for performing statistical analyses - I think there is nothing more to declare.