I hope you can advise me on the choice of a proper statistical test to assess the difference significance between the groups.
I analyze the data from an animal model experiment. I have one control group and three different treatments and one parameter. Unfortunately, there is a large dispersion within the groups and the groups are small. The data do not have a normal distribution (Shapiro-Wilk test). As a result, even if the differences of the means are large by eye (like "Treatment 3" gives twice the value of "Control" and triple the value of the "Treatment 1"), the Kruskal-Wallis test does not indicate the statistical significance of the differences.
The means +/- SD are as follows:
Control: n=6 5659 +/- 5257
Tr1: n=4 3875 +/- 2866
Tr2: n=8 6111 +/- 5823
Tr3: n=7 13413 +/- 7809
BTW. Would the data log transformation help in this case? How do you think about excluding the outliers from the analysis?
Best regards,
Maciej
In my opinion, 1. accepted that result's of using K-W test, no difference among them. 2. replace the outlier with missing value. 3. Transform data with other techniques, for example, log transformation, one of Box-Cox transformations, may be the one you looking for.
If the normal distribution is a bad model for the response, is there a more approprioate model? If so, then you should use the more approriate model.
What such model could be depends on what your response variable is (you did not mention this - but this is the MOST IMPORTANT information about your experiment!). If these are counts (e.g. cell counts), you should use a negative binomial model, if these are concentrations or intensity measures, you should use a log-normal or a gamma model.
But still it may be that the variances are too large to see systematic differences in such relatively small samples.
Excluding outliers because they don't suit your hopes is a no-go. Checking for outliers just helps to identify *possible* problems with samples, so that you can go back and find a reason why there samples might not be trusted. If you don't find such reasons, the outlier provides the very same information like any other value and is possibly the one important observation telling you that the rest is a bit biased.
Often, outliers seem problematic just because you have a particlar (and possibly wery wrong) intuition about how the distribution should look like. Changing your distributional assumptions may make these values way less suspicipous (there is a phrase: "I am not an outlier! - I just haven't found my distribution yet.")
This is a test of the equality of two means. If the samples are sufficiently large the means will approximately follow the normal distribution by the central limit theorem. Their difference shall also follow the normal distribution. Therefore the normal distribution z test will be applicable.
Outliers should be discarded only if there were mistakes in data collection.
Assumption of the normality is one aspect but heteroskedasticity could be worst for the statistical analysis.
Choose the appropriate model, as already Jochen Wilhelm mentioned.
At the end, you could try something like in the attachment (for one-way analysis).
The atypical values can be eliminated through the mahalanobis distance with values of .00
Undoubtedly the data that do not follow a normal distribution to be analysed by Kruskal-Wallis test (since you have one control group and three different treatments groups). Results of Kruskal-Wallis test may be conservative compared to parametric equivalent, but there is no contra indication that this cannot be applied. Even if the differences of the means are large by eye, the statistical insignificance indicates there are other reasons for it. One of the main reasons is less sample size. Excluding the outliers from the analysis will result in much smaller sample size and leads to again the same problem (less sample size). This is the problem with the data from animal model experiments.