Hello Idan,

You can always apply an exact/permutation test, without regard for distributional assumptions. Many statistics packages are including resampling/bootstrap tests as an option (and these tend to yield results very close to exact values, especially as the number of resampling replications is increased).

Here are some links for how to do this:

1. https://bcdudek.net/anova/robust-methods-and-resampling-methods.html

2. http://rcompanion.org/rcompanion/d_06a.html

Do not opt for the Kruskal-Wallis anova by ranks, as that test does require homogeneity.

Good luck with your work.

You have not said how extreme you believe the heterogeniety is, or in what ways the residuals are not normally distributed (e.g., usually this is the kurtosis being higher than the normal). Also, you have not said what your hypothesis is about. The ANOVA is about differences in means. Is this what you want, or just some notion of location? Also, are there lots of ties (e.g., if so, the Qanova function from WRS2 might be a good option). But, the choice will be informed by your answers, and presumably answers to questions other people have before they are confident enough to suggest a procedure.

1. Homogeneous distribution:

Based on the robvar command in STATA these are the results:

W0 = 4.8178100 df(3, 629) Pr > F = 0.00252256

W50 = 3.0713811 df(3, 629) Pr > F = 0.02730491

W10 = 3.6291696 df(3, 629) Pr > F = 0.01284074

2. Normal distribution:

Shapiro-Wilk W test:

W = 0.928

V = 29.833

z = 8.248

Prob>z = .000

Shapiro-Francia W' test:

W = 0.935

V = 29.071

z = 7.505

Prob>z = .000

*Of note, the non-normal shapes are very similar for all four groups.

3. My hypothesis is about differences in means.

4. " ...are there lots of ties?" Is there a test for such issue?

I am not sure about the alternative for ANOVA, but for post-hoc I think wilcoxon signed-rank test should be a good choice

Idan Haklay , I'm not interested in hypothesis tests about homogeniety and normality, say what the different sds are and what the distributions are like. Jochen Wilhelm has lots of great posts on why people should not be concerned with these tests for deciding the appropriateness of using procedures that make certain assumptions. If your hypothesis is only about means, then certain things like transformations (whether on the variables themselves or in a glm), the Kruskal-Wallis and Mood tests, etc., are not appropriate. Have you tried bootstrapping?

Hi if you are dealing with time-to-event data (e.g., reaction times, saccade latencies, fixation durations, etc.) you can use event history analysis: Article Analyzing Response Times and Other Types of Time-to-Event Da...

Daniel Wright The sd's are:

T0 - 6.335

T1 - 6.665

T2 - 8.556

T3 - 6.12

All 4 distributions are left-skewed/negatively skewed