Formal tests of assumptions ara non-sensical.

Three reasons:

1. If they are "significant" it does not tell you if the violations are relevant.

2. if they are "non-significant" you don't know if you miss relevant violations just because the sample size is too small.

3. The normal distribution and other models as well are theoretical *models*, they are not identical to empirical data. There is no question whether or not assumptions are met. They are not. So tests on the assumptions test null hypotheses that are known to be wrong a priori. The question is: are the violations relevant? This question is not anwered by tests.

Maybe a fourth point:

The final analysis (for which the assumptions are required) also make another implicit assumption that is never discussed at all: the underlying model is correct. But this is obviousely impossible to check. A given set of data may be more or less plausible given a model, so one may select a model based on this plausibility of the data, but there is no way to see what would be a correct model.

To answer the original question:

If one has counts, one would start thinking about a Poisson model or a negative binomial model. If I decide to use a normal model anyway, then I need my reasons for this. Why should I then go back and check how plausible the observed residuals look under my carefully selected model?

If one has a strictly positive response, one would start thinking about a gamma-model or a log-normal model. Again, if I decide to use a normal model anyway, then I will have my reasons and the same things apply as for counts.

The problem is that many people do not think about the kind of response they have and what statistical model is appropriate to deal with such data. Instead they always start with the normal model (that's just the standard chice and not a "careful selection") and start getting nervous when someone asks if the assumptions were met. Others try to be smarter and do formal tests on every assumption they are aware of, just to decide based on the observed data if the data needs to be transformed of some different analysis should be performed. When this is all done to eventually get p-values, then this is just all a waste of time, because these p-values do not do anymore what they are supposed to do, i.e. to control an error-rate.

You do need to test for normality and Skewness.

if skewed:

You can transform your data (log, reciprocal, reciprocal square, reciprocal square root) for example log can help remove positive skew, or you could use a non-parametric test such as Kruskall-Wallis test?

Thank you Robert for your answer.,

but in my field, I found nobody does it! Even in the published papers, the skewness test is rarely mentioned! 

Recently, I submitted a paper, and it returned with some comments from a reviewer to add the skewness values of the data, which I wonder if it is really necessary.

Mohammad, If it's a condition you will need to add the skewness and therein lies the issue. The Reviewer/editor may be statistician and would pick up on this. With business and economics data this is big issue as Skewness points to heteroscedasticity which you should not have in your data

Formal tests of assumptions ara non-sensical.

Three reasons:

1. If they are "significant" it does not tell you if the violations are relevant.

2. if they are "non-significant" you don't know if you miss relevant violations just because the sample size is too small.

3. The normal distribution and other models as well are theoretical *models*, they are not identical to empirical data. There is no question whether or not assumptions are met. They are not. So tests on the assumptions test null hypotheses that are known to be wrong a priori. The question is: are the violations relevant? This question is not anwered by tests.

Maybe a fourth point:

The final analysis (for which the assumptions are required) also make another implicit assumption that is never discussed at all: the underlying model is correct. But this is obviousely impossible to check. A given set of data may be more or less plausible given a model, so one may select a model based on this plausibility of the data, but there is no way to see what would be a correct model.

To answer the original question:

If one has counts, one would start thinking about a Poisson model or a negative binomial model. If I decide to use a normal model anyway, then I need my reasons for this. Why should I then go back and check how plausible the observed residuals look under my carefully selected model?

If one has a strictly positive response, one would start thinking about a gamma-model or a log-normal model. Again, if I decide to use a normal model anyway, then I will have my reasons and the same things apply as for counts.

The problem is that many people do not think about the kind of response they have and what statistical model is appropriate to deal with such data. Instead they always start with the normal model (that's just the standard chice and not a "careful selection") and start getting nervous when someone asks if the assumptions were met. Others try to be smarter and do formal tests on every assumption they are aware of, just to decide based on the observed data if the data needs to be transformed of some different analysis should be performed. When this is all done to eventually get p-values, then this is just all a waste of time, because these p-values do not do anymore what they are supposed to do, i.e. to control an error-rate.

In addition to all the excellent points Jochen raised, using a rank-based test in place of a t-test (or ANOVA etc) to deal with heterogeneity of variance or skewed outcome variables is not the cure-all that many people think.  See, for example, the simulation study reported by Fagerland & Sandvik (2009) linked below.  As their study shows, small differences in population variance or skewness cause inflation of the observed Type I Error rate (when the Mann-Whitney U test is used in place of the t-test), and the inflation gets worse as the sample size increases. 

HTH.

Article The Wilcoxon-Mann-Whitney test under scrutiny

The assumption of ANOVA is that the residuals are normally distributed, not necessarily that the data are normally distributed (as is commonly stated). It is possible to have non-normal data that result in normally distributed residuals that meet the assumptions of the ANOVA model.

I agree with Jochen. Although it is important to check your assumptions, formal hypothesis tests are not the way to do this.

Firstly, a preliminary test alters the significance level of the research hypothesis and affects confidence coefficients. Therefore your published significance levels are no longer what they are stated to be.

Secondly, assumptions are never exactly true and a large enough sample will reject even small departures, while a small sample will often fail to reject large, important, departures. But the statistical methods for your main research question are often robust enough to tolerate moderate departures from your assumptions. For example ANOVA usually assumes equal variances and is quite robust to departures from this assumption so long as the sample sizes are not too different. This has been tested by simulation. However, the F-test for two equal variances is not robust to non-normality. Doing such a test before a two sample t-test has been likened to sending a dinghy out in a storm to test whether it is safe for an ocean liner.

Moderate departures from the assumptions also affect sigificance levels, but for a robust test like ANOVA the effect will be small.

Therefore informal checks for your assumptions are sufficient. Normal probability plots will reveal long tails that might be handled with a transformation or by non-parametric methods. Variance ratios between 0.5 and 2 suggest that ANOVA with the equal variance assumption is OK if the samples sizes are nearly equal.

Thank you all for the valuable contributions. 

I have some data which I analyzed by 'Statistix v10' software. The basic statistics are shown in the attached picture below. I took these results, calculated the maximum allowable skewness {2 * standard error of skewness (ses), http://j.mp/1Dvwy7c} which I placed in row 20. then compared the absolute skewness to the max. skew, to determine the significance of the skewness (row 21).

My question is: To what extent could I forgive the skewness value? as almost all my data seemed skewed!

Nb: After I tried the logarithmic transformation (LN), most of the variables are not skewed.

Graphical displays would be helpful here. It is good advice to always graph your data. It does seem that the assumptions for the usual ANOVA are not satisfied which is probably why a referee queried the skewness. Given the wide range of standard deviations, he might also have queried the equal variance assumption.

As the skewness varies from positive to negative, a log transformation which improves one column, would make others worse. I would suggest non-parametric methods.

But graph the data and their logs. Apart from the overall shape, this will help you detect any outliers that could affect the standard deviations.

The analysis of variance does not assume that the data are normally distributed.

The assumption is that the errors, after fitting the model, are normally distributed.

Fit the ANOVA model and the check the distribution of the residuals as these estimate the errors.  It is possible to formally test these for skewness and kurtosis but simple visual inspection of the standardized residuals plotted against the predicted values is most helpful.

Hope this helps...

In addittion to all the excellent comments above. I strongly advice to use graphical exploration to test your assumptions. You could also generate a "ideal" data-set, e.g. same mean and SD than your original data and see how the analyses comes out for that data-set. Finally, if you deal with a few Outliers cou re-analyze with and wirhout these point and see whether your results change. All this to understand your data, potential bias and to make sure you draw sound conclusions in the end.

I had just written a blog post on the t-test and the myths surrounding it, but I see from the excellent replies above that I shall have to write a follow-up post tackling even more of the myths! 

Anyhow, my two centimes worth is here:

http://bcss1.blogspot.ie/2015/02/myths-and-nonsense-about-t-test.html

http://bcss1.blogspot.ie/2015/02/myths-and-nonsense-about-t-test.html

PS.  If you have a treatment effect, your data will almost certainly NOT be normally distributed.

SALAM

I THINK FOR SMALL SAMPLE SIZE THIS CONDITION REQUIRES,  FOR LARGE SAMPLE IN EACH GROUP WE HAVE NO NEED TO CHECK NORMALITY TEST.

The relevant assumption for a linear model (ANOVA is one kind of linear model) is that the residuals are normally distributed, not the raw data. Other points raised above, re: real data never fitting a theoretical distribution, are excellent.

Dennis and Mollie are correct, assuming each population is normally distributed, if the means are different the combined distribution is a mixture of normals with different mean. If the means are far apart the combined distribution could have multiple modes. If the means are the same but the variances differ, or if both differ we also have a mixture. Their point is that this doesn't mean that we can't use the normality assumption because it is not normality of the combined sample that matters. My original point was that we should not be too fussy about normality, especially with large samples.

Side by side box and whisker plots may be used as a rough check on the assumption of equal variances, and to some extent on normality.

I agree with Ronan, the Mann-Whitney test (and its generalisation, the Kruskal–Wallis test) is a test that the populations are identical (same location and spread and same shape). Either means or variances can lead to a significant difference. The same applies to ANOVA. Even for tests that are most sensitive to differences in location, you should be cautious whenever variances and shapes differ by a large amount.

I look forward to reading more of the myths. I hope to read that ANOVA is very robust to departure from normality. Or possibly that the Kruskal–Wallis test is more robust than we might have feared.

You can use other type ANOVA like Kruskal-Wallis etc. as said above. However, it works for big enough sample sizes.

As Audrias says, it works for big enough sample sizes. But how big is big enough? The sample sizes of 288 should be big enough, but plots of the data are stll important to understand the data. Statistical methods are not a machine to be used blindly. Outliers inflate variances. Are outliers a symptom of a long tailed distribution? Or are they mistakes? Or is there some explanation worth investigating for its own sake?

Also what is the effect of the very different standard deviations for the samples? ANOVA treats them as equal. The test is robust, but is it robust enough?

If only the different variances are the concern: wouldn't it solve the "problem" (if there is one at all because of the robustness) when a weighted linear model was used?

I think data transformation can be an option if the data does not follow normality. You may use Box cox transformation or log transformation and them perform you ANOVA on the transformed data. But you conclusion will be on the transformed data rather than the original one. I am not in the agriculture field, however I use this transformation technique in hydrology and it is common in hydrology. See the link below for an application of such transformation.

https://www.researchgate.net/publication/258225101_Geostatistical_analysis_using_GIS_for_mapping_groundwater_quality_case_study_in_the_recharge_area_of_Wadi_Usfan_western_Saudi_Arabia

Article Geostatistical analysis using GIS for mapping groundwater qu...

Jochen's idea seems to make sense but I'm not quite sure how to choose the weights. However, if there are only two samples (and normal distributions), the problem is known as the Behrens-Fisher problem. This problem is still unsolved.

• https://en.wikipedia.org/wiki/Behrens%E2%80%93Fisher_problem

gives a good discussion. The Welch test is one approximate solution.

The generalisation to ANOVA also has approximate solutions. The Games‐Howell test is a generalisation of the Welch test. I'm not sure of the details but see

• http://www.sagepub.com/field4e/study/smartalex/chapter11.pdf
• http://www.gcf.dkf.unibe.ch/BCB/files/BCB_10Jan12_Alexander.pdf.

Alternatively, if you blindly use ANOVA you are really testing

• H0: all means equal, all variances equal, all distributions normal.

Rejection implies at least one of these is false. Given the large sample sizes, the normal distribution is less important (in the sense that the actual distributions and normal distributions with the same mean and variance would give similar results), so you can assume either a difference between means or between variances. Even if you can assume equal variances, you still want to understand where the differences lie. So the test is not the end point of your research.

Going back to the distribution, even if this doesn't matter, it is still good practice to draw the graphical displays even if you don't publish them. They are important diagnostics.

My sugestion is to use non-parametrical ANOVA. Kruskal-Walis test for example.

Terry,

when the variances are not equal, the ANOVA gives p-values that are not uniformly distributed under the null. The distribution function is a bit "sigmoid", having "too many" p-value close to 0 and also "too many" p-values close to 1, and "too few" p-values in the middle. For NP-testing this means that the nominal alpha level is not held (the Anova is a bit too liberal). However, this is (1) not a very severe problem and (2) is only a problem in NP-testing. If a p-value is assessed as a measure for the statistical significance of the data it is easy to be a bit more careful with the interpretation when the p-value is not *very* close to 0.

The major problem is that relevant differences between some groups are simply obscured by the large variance in another group. In a NP-testing this would mean that the nominal power would not be achieved - given the Anova is done to identify differences between groups and not (as it is actually indended!) to decide between two models (with and without the respective predictor(s)).

I do agree with Jochen on (all) their answers to this topic. In addition to the lack of power in such cases (violation of normality), even when ANOVA is robust enough to that violations (see refs below), it is very important to understand what is happening with that factors (or levels). For example, trying to explain that behavior (the skewness) is it due to some specific cause, and that cause can be explained in some way? In the case of violation of homoscedasticity: Why some level have bigger variances than the others? Why some levels are skewed than the others? Those answers can help to understand the possible lack of effect, that affects the ANOVA on such cases.

Some comments on the violations of ANOVA assumptions:

Normality: ANOVA is robust enough under lack of normality (Glass et al., 1972; Harwell et al., 1992; Lix et al., 1996). 

Homoscedasticity: the lack of homoscedasticity affects ANOVA. In this case, the main problem is not the comparison of the means, in case of heteroscedasticity then you must find an explanation of such variability: Why the variance of one group/level is greater than the others? The ANOVA can be performed (using an alpha of 0.01 due to the heteroscedasticity) if the experiment is well replicated (n>30 and on balanced data). (Box, 1954; Box and Andersen, 1955; Scheffé, 1959; Miller, 1986)

The alternative non-parametric will not explain the heteroscedasticity, and it will be more conservative that the ANOVA (even changing the alpha) (Akritas, 2004; Box and Andersen, 1955; and so on...)

References:

Glass, G.V., P.D. Peckham, and J.R. Sanders. 1972. Consequences of failure to meet assumptions underlying fixed effects analyses of variance and covariance. Rev. Educ. Res. 42: 237-288.

Harwell, M.R., E.N. Rubinstein, W.S. Hayes, and C.C. Olds. 1992. Summarizing Monte Carlo results in methodological research: the one- and two-factor fixed effects ANOVA cases. J. Educ. Stat. 17: 315-339.

Lix, L.M., J.C. Keselman, and H.J. Keselman. 1996. Consequences of assumption violations revisited: A quantitative review of alternatives to the one-way analysis of variance F test. Rev. Educ. Res. 66: 579-619.

Akritas, M.G. and Papadatos, N. (2004). Heteroscedastic one-way ANOVA and lack-of-

fit test. J. Amer. Statist. Assoc. 99, 368-382.

Box, G. E. P. and Anderson, S. L. (1955). Permutation theory in the derivation of robust

criteria and the study of departures from assumption. J. Roy. Statist. Soc. Ser. B

17, 1–26.

Miller, R. G., Jr. (1986). Beyond ANOVA: Basics of Applied Statistics. John Wiley,

New York. (Reprinted 1996, Chapman & Hall, New York.)

Scheffé, H. (1959). The Analysis of Variance. John Wiley, New York

What test can we use when normality and/or homogeneity of variance has failed? - ResearchGate. Available from: https://www.researchgate.net/post/What_test_can_we_use_when_normality_and_or_homogeneity_of_variance_has_failed [accessed Feb 19, 2016].

Yes, of course you should measure skewness before applying any statistical tests. but current skewness measurement methods are sensitive to sample size. We developed a new methodology to measure skewness, irrespective of sample size. j.ponte.2017.2.34