How did you check for normality? Within each cell of your nXm ANOVA or overall? Did you use a test or did you also a visual inspection? If also visual inspection, what caused non-normality? Did you check for outliers? How large is the sample size within each cell?

Without additional information, we are not able to give a well-founded answer. Please show us your plots and data. In essence, there are a lot of things that can be done wrong, in the first place. Please have a look at the slides by Bruce Weaver, to get an impression Presentation Silly or Pointless Things People Do When Analyzing Data: 1. ...

Not all "statistical" deviations from normal are harmful. Under some circumstances, ANOVA can be regarded as robust. There are robust alternatives, so no need to switch immediately (if at all) to non-parametric tests, they are not always the best solution. Sometimes your dependend variable is not suitable for a simple linear model, e.g. count data, bounded data (reaction times for example), so that you might need a generalized linear model (poisson regression, gamma regression...). It depends on your data structure.

Dear Rainer Duesing

thank you for your answer. I am using R and I checked both visually and using Shapiro-Wilk normality test. I have a small dataset (24 soil samples, please see below) half of them had a treatment applied, the others not). The second "factor" of my two-way analysis is due to the fact that these soil samples were treated with 3different manures types+untreated samples.

I would like to assess differences between the treatments and digestate versus no digestate application (on PH - let's leave OM out for now).

number dig-nondig treatment OM PH

1 CTR C50 7.4 7.5

2 CTR C50 7.2 7.7

3 CTR C50 7.7 7.7

4 CTR FMY 5.6 7.9

5 CTR FMY 6.0 7.8

6 CTR FMY 5.6 8.0

7 CTR CM 6.1 7.7

8 CTR CM 5.6 7.4

9 CTR CM 6.0 8.0

10 CTR U 5.7 7.7

11 CTR U 5.8 8.0

12 CTR U 6.0 7.8

1 AD C50 7.4 8.0

2 AD C50 7.6 7.7

3 AD C50 6.0 8.1

4 AD FMY 6.1 7.8

5 AD FMY 6.2 8.0

6 AD FMY 6.1 7.9

7 AD CM 5.8 8.0

8 AD CM 6.2 7.9

9 AD CM 6.0 8.0

10 AD U 5.6 7.9

11 AD U 6.5 8.0

12 AD U 5.8 8.0

You did not answer the questsion, how you checked for normality: did you just check your dependent variable (do you have two DV?) for normality or within each cell or the residuals?

Sorry! Residuals which I calculated as Xi - mean(X).

mean(X) is the overall X, so grand mean?

The mean of the pH values, yes! While Xi is the single value of the dataset to which I subtracted the mean

But these are not the residuals! You just have your original DV/distribution and shifted it by the grand mean! This makes no sense. You have to assess normality within each group or with the residuals. Residuals are the single values minus the model, not the grand mean. In your case, you have factor A with i levels, B with j levels and k observations. Therefore, the residuals would be the Xijk values minus the mean value of the i,j group, i.e. mean(Xij) NOT mean of all values.

I dont know how you calculated your ANOVA in R, but I am pretty sure you can call the residuals directly from your output (I would use the lm function).

Besides that: I would have issues with the validity of your results. 3 observations for each cell is very, very small. I assume you did not do a power analysis? With only 3 abservations, even the slightest deviations from the true value may cause a significant effect or cover a true effect. I am not sure if you should do an inferential analysis at all. Why not describe your results descriptively?

Ok, I just checked, if you use the aov command, it works as the lm command, and you can easily plot all checks and also the residuals.

Thank you very much for your suggestions, I really appreciate your explanation. Maybe I could follow your proposal and describe the results instead of performing the analysis with this dataset as I have little number of observations. However I tried the aov in R!

May I still ask if there is a valid alternative to the two-way anova as a non parametric test? Not in this case, and I understand this will depend upon the data of course, but is there any? Beside Friedman test ?

I think I understood better now.

I built the model with "aov".

I saved the residuals

aov_residuals shapiro.test(x = aov_residuals )

p-value = 0.7448

Thank you!

Again a question: for what do you need a non-parametric test? Please don't mix up "non-parametric" with "assumption-free", this is not the same thing. There are also assumptions for non-parametric tests, but somehow people treat them as there were none. Additionally, the do not test the same hypotheses as for example an ANOVA.

Therefore, I cannot recommend any specific tests. I would decide case by case what to do. Check all assumptions for my linear model and react to the specific violations. Heteroscedasticity: use robust SE estimators or use WLS. Normality: check the data generating process, maybe use generalized linear model, handle outliers, use robust versions of the ANOVA, with robust estimators (trimmed means, M-estimator; see the books by Rand R. Wilcox for tons of suggestions and R functions for all of them!!), maybe switch to Bayesian inferencing. There are lots of possibilities.