In my research, I am comparing 7 groups with equal sample size (20 each). But most of the response variables did not meet the assumptions of normality and homogeneity of variance. Can I still use One-way Anova or can you suggest me alternative non-parametric test?
Good question!
Is any method valid if the data (i.e., reality) don't meet the assumptions underlying the validity of the method?
In engineering, theoretical inadequacy and resulting design flaws (e.g., in design/construction of bridges, aircraft, rockets, and such) are sometimes very obvious. In soft science, only accumulating failures to replicate or to extend findings over the passage of time (i.e., as opposed to the "one-and-done" failure criterion that is the standard for hard science) seem to move once-popular methods--including legacy parametric statistical paradigms--into the history books.
My question is rhetorical, and the fact that articles featuring interpretation of invalid statistical conclusions permeate the literature helps to explain why ever more journals are attacking and forbidding the exclusive use of p-values in results sections of empirical articles. Increasingly, emphasis is being placed on the strength of the effect, as well as on the cross-generalizability of the effect when the model is used to classify independent random samples.
Unfortunately, in addition to questions regarding the validity of their statistical conclusions, ANOVA and regression-based models also have very poor predictive accuracy, even for very simple metrics and designs--though this can be rectified to some extent:
- http://optimalprediction.com/files/pdf/V2A3.pdf
In order to obtain statistical models that identify solutions that explicitly maximize the ability of the model to make accurate predictions concerning the actual score (class status) of individual observations, one has to use maximum-accuracy methods that explicitly accomplish this specific mathematical optimization. Not only do these models maximize predictive accuracy, but they are more parsimonious than parametric models, and cross-generalize well.
- https://www.researchgate.net/publication/291947229_Using_data_mining_techniques_to_characterize_participation_in_observational_studies
Perhaps most important for your specific question, the p-values for this non-parametric method are obtained as permutation probabilities that require no distributional assumptions--the p-values are always valid.
- https://www.researchgate.net/publication/288834108_Maximizing_Predictive_Accuracy
Regarding your analysis, here are two introductory articles about a non-parametric alternative (maximum-accuracy range test) to one-way ANOVA:
- http://optimalprediction.com/files/pdf/V2A30.pdf
- http://optimalprediction.com/files/pdf/V2A32.pdf
And, in the following article a multidimensional maximum-accuracy range test (known as "structural decomposition") is used to identify structure in and compare ratings between a multivariate array of different symptoms assessed for a single-case time-series:
- http://optimalprediction.com/files/pdf/V2A25.pdf
Not to short-shrift cross-generalizability, the following article shows that analyses that ignore cross-generalizability and focus only on maximizing performance in the training sample can find models that do well in training, but that replicate at chance (or worse) levels in attempted replication or extension.
- http://optimalprediction.com/files/pdf/V5A1.pdf
The myopic focus on training performance is, in part, clearly a factor influencing the non-reproducibility crisis that has been noted for several years--that is, for several research funding cycles--in the US Congress.
Additional examples of these and other related methods (e.g., factorial ANOVA, multiple regression) versus maximum-accuracy methods are available at:
- http://ODAJournal.com
Book Maximizing Predictive Accuracy
Article Using data mining techniques to characterize participation i...
Homogeneity of variance is a more important assumption of one-way ANOVA. Since you have all cell sizes equal, violating homogeneity assumption is less severe.
But another thing to notice is that it is the residuals that must be normal and homogeneous not the data itself. If your residuals satisfy these assumptions you can go with ANOVA.
For a more robust test, you can use Welch's ANOVA for non-homogeneous case. Actually some authors argue that it should be used any way.
A nonparametric candidate is Kruskal-Wallis test. You can compare the results of these three tests. I would personally go for Welch test in most situations.
I like Mehmet suggestions. However, violation of normality may not be case once you have corrected for heteroskedaskicity. You can do Anova as a regression without an intercept. Since you have 7 groups then create an identifying dummy for each group then treat all the scores as DV and D1 D2 D3 D4 D5 D6 D7 as IVs then run. In Eviews,
look for Newey-West Bandwidth and Automatic selection. Note the F obtained is your ANOVA F. This shouldn't take 10minutes to do. That's all you need.
Note that you can use C D1 D2 D3 D4 D5 D6 if you are not familiar with how to estimate without intercept term.
Good question!
Is any method valid if the data (i.e., reality) don't meet the assumptions underlying the validity of the method?
In engineering, theoretical inadequacy and resulting design flaws (e.g., in design/construction of bridges, aircraft, rockets, and such) are sometimes very obvious. In soft science, only accumulating failures to replicate or to extend findings over the passage of time (i.e., as opposed to the "one-and-done" failure criterion that is the standard for hard science) seem to move once-popular methods--including legacy parametric statistical paradigms--into the history books.
My question is rhetorical, and the fact that articles featuring interpretation of invalid statistical conclusions permeate the literature helps to explain why ever more journals are attacking and forbidding the exclusive use of p-values in results sections of empirical articles. Increasingly, emphasis is being placed on the strength of the effect, as well as on the cross-generalizability of the effect when the model is used to classify independent random samples.
Unfortunately, in addition to questions regarding the validity of their statistical conclusions, ANOVA and regression-based models also have very poor predictive accuracy, even for very simple metrics and designs--though this can be rectified to some extent:
- http://optimalprediction.com/files/pdf/V2A3.pdf
In order to obtain statistical models that identify solutions that explicitly maximize the ability of the model to make accurate predictions concerning the actual score (class status) of individual observations, one has to use maximum-accuracy methods that explicitly accomplish this specific mathematical optimization. Not only do these models maximize predictive accuracy, but they are more parsimonious than parametric models, and cross-generalize well.
- https://www.researchgate.net/publication/291947229_Using_data_mining_techniques_to_characterize_participation_in_observational_studies
Perhaps most important for your specific question, the p-values for this non-parametric method are obtained as permutation probabilities that require no distributional assumptions--the p-values are always valid.
- https://www.researchgate.net/publication/288834108_Maximizing_Predictive_Accuracy
Regarding your analysis, here are two introductory articles about a non-parametric alternative (maximum-accuracy range test) to one-way ANOVA:
- http://optimalprediction.com/files/pdf/V2A30.pdf
- http://optimalprediction.com/files/pdf/V2A32.pdf
And, in the following article a multidimensional maximum-accuracy range test (known as "structural decomposition") is used to identify structure in and compare ratings between a multivariate array of different symptoms assessed for a single-case time-series:
- http://optimalprediction.com/files/pdf/V2A25.pdf
Not to short-shrift cross-generalizability, the following article shows that analyses that ignore cross-generalizability and focus only on maximizing performance in the training sample can find models that do well in training, but that replicate at chance (or worse) levels in attempted replication or extension.
- http://optimalprediction.com/files/pdf/V5A1.pdf
The myopic focus on training performance is, in part, clearly a factor influencing the non-reproducibility crisis that has been noted for several years--that is, for several research funding cycles--in the US Congress.
Additional examples of these and other related methods (e.g., factorial ANOVA, multiple regression) versus maximum-accuracy methods are available at:
- http://ODAJournal.com
Book Maximizing Predictive Accuracy
Article Using data mining techniques to characterize participation i...
I have no idea about Newey-West Bandwidth and Automatic selection (if they are any better for violation of assumptions) but I can say that standard regression analysis will not give a different result from ANOVA.
Robust regression might give a different result and worth trying though.
I am happy that Mehmet agrees with me that "standard regression will not give a different result from Anova." The F obtained are exactly the same.
Given that she noted the presence of heteroskedasticity and possible non-normality
it makes sense to use an estimator that gives a consistent and reliable F- value for the null hypothesis that the means are all equal . After applying the Newey-West estimator, she will obtain an F-value she can report.
Some of the discussions are very interesting.I really appreciate your help. Thanks for the very insightful explanations!
- Badges
- Science topic