Chris, 

I would check out these sources to see what recommendations they might have.

Glass, G.V., P.D. Peckham, and J.R. Sanders. 1972. Consequences of failure to meet assumptions underlying fixed effects analyses of variance and covariance. Rev. Educ. Res. 42: 237-288.

Harwell, M.R., E.N. Rubinstein, W.S. Hayes, and C.C. Olds. 1992. Summarizing Monte Carlo results in methodological research: the one- and two-factor fixed effects ANOVA cases. J. Educ. Stat. 17: 315-339.

Lix, L.M., J.C. Keselman, and H.J. Keselman. 1996. Consequences of assumption violations revisited: A quantitative review of alternatives to the one-way analysis of variance F test. Rev. Educ. Res. 66: 579-619.

I wouldn't be as dismissive yet about abandoning ANOVA.  It would depend how far out of whack your data is from normality.  There actually isn't a terrible increase in false positive rate.  Have you checked such tests as Komolgorov-Smirnoff to check that it is significantly different from normal first?  My guess would be that if it would not be significantly different there, then you would be fine to use ANOVA.

Thanks Matthew.  I've not read the two papers you mention, so I'll give them a read.  I had read about the robustness of ANOVA though.

I used Shapiro-Wilk, Levene and Fligner tests and had multiple cases where the assumptions weren't met (P < 0.001) .  However, looking at the results a factorial ANOVA and the results I'm seeing from Mann-Whitney U comparisons, I think the ANOVA's  handled the failed assumptions well. 

Since I'll end performing the planned comparisons anyway, I'm wondering whether it's acceptable to simply forfeit the ANOVA altogether?   I'd appreciate your thoughts and thank you for the information provides.

It sounds like voting is the dependent variable, and is binomial (X or Y). In this case you might look at logistic regression, where independent variables can include sex, age group, personality, and their interactions.

Hi Salvatore,  I've looked at logistic regression and may yet go back to it (I'm using it in another study).  What I liked about having the personality trait as the DV, was I felt it was simpler for people to see the differences between voters visually (e.g. a box plots or similar).  Going this route, I could also annotate the plots to highlight which comparisons were significant. 

I'll reconsider logistic regression though.  Thanks for providing your input, very much appreciated.

Yes, you could do simple tests of association, using Personality as DV (Mann-Whitney for Sex and Vote, and maybe Kendall correlation or Kruskal-Wallis for Age).

But it is often better if you can put all variables together in one model.  This is especially true when you have unbalanced data.  That is, if most of voters for A happen to be Male in your population, by reporting the Personality for voters of A and B, your results are conflating the effects of Sex and Vote.

You'll often hear results for something as being "adjusted for sex"  or "adjusted for age".  One way to accomplish this is to put all the variables in one model, and report least square means instead of arithmetic means.  A simple example is here:  http://rcompanion.org/handbook/G_05.html .  Each L.S. mean is "adjusted for" the other independent variables in the model.

*     *     *

Here's my advice:

1) Make sure you understand the assumptions of a common general linear model (e.g. linear regression or two-way anova).  The data should be _marginally_ normal.  Usually you assess this by asking if the _residuals_ are normal, not the data itself. And often including certain interactions in the model can make the residuals better behaved.

2) Confirm that your DV is continuous and that common general linear model is the right approach.  If the DV is e.g. proportion, Likert, count, etc., you want to use a different modelling approach.

Also, I strongly recommend against uses statistical tests for assessing normality and homoscedasticity.  Plot a histogram, q-q plot, or residuals vs. predicted value.

Your eye is a better judge.

See: http://blog.fellstat.com/?p=61

And my description of assessing model assumptions here: http://rcompanion.org/handbook/I_01.html#_Toc459550775