It is more effective and efficient to use 2-factorial ANOVA because it allows you to look at both main effects (1. gender, 2. treatment) and the potential interaction effect (gender*treatment) in a single step. You obtain F statistics for all three effects that you can easily interpret in your 2 x 2 design. The F statistic for the main effect of gender tells you whether overall (i.e., regardless of treatment), there is a gender difference in means. The F statistic for treatment tells you whether overall, there is an effect of treatment (regardless of gender). The F statistic for the interaction gender*treatment tells you whether the treatment differs in efficacy between males and females.

No - adding additional factors (or levels) to a model will not result in the same outcome because it becomes an entirely different model.

What are the sample sizes in each of the 4 groups? Are they roughly equal and do they satisfy the equality of variance assumption?

In sum, the more factors and/or levels you add to a LM, the larger the sample size you will need to achieve sufficient power to detect differences

Ok, first of all, thank you all very much for your insight!

So, when comparing a 1x4 vs a 2x2 ANOVA, the 2x2 ANOVA has more relevant information, as it not only test both independent variables but also there interaction effect. For example, I could get a significant or non-significant effect of treatment (averaged over all samples, no matter the gender) while simultaneously having a significant or non-significant effect of gender (averaged over all groups no matter if they belong to the control or treatment group)? Then it would also test the interaction of gender*treatment between all groups?

Now, as for post hoc test, Jochen Wilhelm

For example, in an experiment I performed, I got a significant effect of treatment, but not of gender or interaction between those. I did do both, "a priori" and "post-hoc" testing to compare those and maybe understand them better. When comparing all groups with each other ("post-hoc") and correcting with Tukey method, I dont get significant effects in any comparison. However, when I compare only the groups I am logically interested in, so basically the "simple effects" you mentioned before (treatment effect in males, and treatment effect in females /or sex effect in untreated, and sex effect in treated), I get a significant effect between the male-ctr vs male-treatment group. Can I robustly argue, that this is a statistically significant effect, even though a "post-hoc" test would suggest otherwise?

Hello Stefan Braun . Your question reminded me of a little demo I cobbled together a few years ago. It starts by estimating a conventional 2x2 ANOVA model (via the UNIANOVA command in SPSS). It then estimates a one-way ANOVA model on the same data via the ONEWAY command. But the ONEWAY command includes a bunch of /CONTRAST sub-commands. Two of them duplicate the main effect tests from the two-way ANOVA, and a 3rd duplicates the interaction test. The others provide tests of all of the simple main effects. As I said, I used SPSS, but the same thing could be done using any statistical software. I hope you find the demo helpful. I'll attach both the syntax and the output, the latter as a PDF so anyone can view it.

Cheers,

Bruce

Jochen Wilhelm