I use Two-Way Mixed ANOVA with 3 levels of between-subjects factor. I have no significant interaction, so I won't testing pairwise comparisons for one factor within each level of other factor. I have significant main effect of between-subjects factor. So I need to run post hoc tests. This menu in SPSS contains tests for equal variances (e.g. Tukey's) and for unequal variances (e.g. Games-Howell) like one-way ANOVA. But output of mixed ANOVA includes three separate Levene's tests for each level of within-subject factor only, that corresponds with simple main effects, not with main effect of between-subject factor. How can I choose post hoc tests for main effect of between-subject factor if I have not overall Levene's test?
You really need to use a procedure which handles your model. For example emmeans in R or SAS. I'm sure there's an analogous procedure in SPSS, but it may not be in the gui menus.
Salvatore S. Mangiafico . SPSS has EMMEANS procedure, but there are only 3 correction for multiple comparisons, which are compatible with this option - least significant different (too liberal), Bonferroni and Sidak (conservative). Post Hoc... menu includes better alternatives such Tukey's or Games-Howell's test, but choosing among them based on equal or unequal variances.
Hello Aleksander. First, for a factor with 3 levels, Fisher's LSD does control the family-wise alpha at the same level as the per contrast alpha, and has more power than the other procedures you might choose. If you need references to support this claim, see Statistical Methods for Psychology by David C. Howell and Serious Stats by Thom Baguley (among others).
Second, if the sample sizes for the 3 groups are the same (or nearly so), ANOVA is very robust to variance heterogeneity, and I would certainly not base my decision on whether Levene's test (or any other test of variance heterogeneity) was significant or not. Here are two relevant excerpts from Howell’s Statistical Methods for Psychology (6th Ed., p. 316).
"As we have seen, the analysis of variance is based on the assumptions of normality [within groups] and homogeneity of variance. In practice, however, the analysis of variance is a robust statistical procedure, and the assumptions frequently can be violated with relatively minor effects. This is especially true for the normality assumption."
"In general, if the populations can be assumed to be symmetric, or at least similar in shape (e.g., all negatively skewed), and if the largest variance is no more than four times the smallest, the analysis of variance is most likely to be valid. It is important to note, however, that heterogeneity of variance and unequal sample sizes do not mix."
Howell, D. C. (2009). Statistical methods for psychology. Cengage Learning.
And here is an article with a humorous title that pays tribute to George Box's famous quip about testing for heterogeneity of variance.
- https://riel.cdu.edu.au/sites/default/files/managed/downloads/McGuinness-AustralEcology2002-ANOVA_0.pdf
HTH.
It seems to me that if you fit your model with the MIXED command, you will want to use the EMMEANS subcommand within that command. Using a straight Tukey's test or Games-Howell really doesn't match your analysis.
Another option for adjustment is to get the unadjusted p-values (LSD), and then adjust these manually. For example, the Benjamini & Hochberg (BH) adjustment controls for False Discovery Rate (FDR), and so can be much less conservative than the Familywise Error Rate methods (FWER) mentioned. The code below could be used at the webpage below to run p-value adjustments in R without installing anything. (Or see this page for instructions in SPSS: https://www.researchgate.net/post/How_can_I_calculate_false_discovery_rate_using_spss).
I'm also including a plot of some adjustment methods for a set of 28 pairwise comparisons. With this many comparisons, the FWER methods (Bonferroni, Sidak, Tukey-adjusted [which is very close to "mvt" in this case]) are quite conservative. BH does adjust up the p-values relative to LSD, but not by too much at this scale.
https://rdrr.io/snippets/
Pval = c(0.01, 0.5, 0.001, 0.007, 0.1)
p.adjust(Pval, method = "BH")
Hi Salvatore. When Aleksandr says mixed ANOVA, I would not infer that he means he used the MIXED command in SPSS. People in Psychology typically use "mixed (design) ANOVA" to describe a classical ANOVA model with at least one between-subjects factor and at least one within-subjects (or repeated measures) factor. So my guess would be that Aleksander is using the GLM command (Analyze > General Linear Model > Repeated Measures in the GUI). But like MIXED, GLM Repeated Measures does have the EMMEANS sub-command with the same options.
- https://www.ibm.com/support/knowledgecenter/en/SSLVMB_25.0.0/statistics_reference_project_ddita/spss/advanced/syn_glm_repeated_measures.html
Although the documentation linked above does not show it, one can also use the POSTHOC sub-command to get the usual array of multiple comparison procedures SPSS offers for any between-subjects effects in the model. You can see the list of available procedures here:
- https://www.ibm.com/support/knowledgecenter/en/SSLVMB_25.0.0/statistics_reference_project_ddita/spss/advanced/syn_glm_univariate_posthoc.html
As I said before, with 3 groups, I would use Fisher's LSD (unless the group sizes varied a lot and the variances were heterogeneous).
HTH.
- Badges
- Science topic
- Similar topics
- Mathematical Sciences
- Statistics
- Statistical Software
- SPSS