Todd, as fas as I know you are wrong here: ". It is a convention (perhaps a prejudice) in statistics to start with main effects, and to only proceed to testing interactions if the main effects are significant."

The opposite is true: If there is a significant interaction, the interpretation of main effects is not straight-forward. In fact, the main effects can be "non-significant" but there is a strong and relevant interaction. Here is a simple example*,**:

Response: Happiness (let it be measured on a ratio scale; 0=neutral, 0 happy)

Factor A: Gender (female/male)

Factor B: Room color (pink/blue)

Results:

A B response

female pink +2

female blue -2

male pink -2

male blue +2

Here, both main effect (gender as well as color) are 0 (and "non-significant"), but there is a very strong interaction of gender x color (4 arbitrary units of happiness).

Going your way, one would conclude that neither color nor gender are related to happiness, so it would not matter in what color you should paint the room for your daughter or son. But in fact your son would be happier in a blue room, whereas your daughter would be happier in a pink room.

*based on a relatively common (stupid or not) prejustice that gils prefer the color pink over blue and boys prefer blue over pink.

** it is just an example! It may not be sensible beyond its mathematical aspects.

PS: If you have a son and a doughter but only one room, paint it violet... ;-)

Yes, it is certainly possible for parts of a model to be "significant" while the entire model is not. For instance, suppose X explains Y (e.g., "significant" association). It is possible to add to the model (starting with X) parts of no relevance so that ultimately the explanatory part of the model (the X part) is washed out. This because the threshold for the global test of the model is increasing with each additional variable because of increasing degrees of freedom.

Your quandary about main effects and interactions is interesting, but not unusual. It is a convention (perhaps a prejudice) in statistics to start with main effects, and to only proceed to testing interactions if the main effects are significant. Therefore, if results showed no significant main effects, many of us would leave it at that and report null findings. However if there was a compelling reasons to investigate the interactions, we might proceed to look at the interactions anyway. A significant results for one of the interactions can easily be dismissed as type 1 error due to multiple testing, unless it was highly significant (e.g., meets a Bonferonni threshold). If your global test (e.g., test of the main effects and interactions simultaneously) is non-significant, then this is likely a null result.

Todd, as fas as I know you are wrong here: ". It is a convention (perhaps a prejudice) in statistics to start with main effects, and to only proceed to testing interactions if the main effects are significant."

The opposite is true: If there is a significant interaction, the interpretation of main effects is not straight-forward. In fact, the main effects can be "non-significant" but there is a strong and relevant interaction. Here is a simple example*,**:

Response: Happiness (let it be measured on a ratio scale; 0=neutral, 0 happy)

Factor A: Gender (female/male)

Factor B: Room color (pink/blue)

Results:

A B response

female pink +2

female blue -2

male pink -2

male blue +2

Here, both main effect (gender as well as color) are 0 (and "non-significant"), but there is a very strong interaction of gender x color (4 arbitrary units of happiness).

Going your way, one would conclude that neither color nor gender are related to happiness, so it would not matter in what color you should paint the room for your daughter or son. But in fact your son would be happier in a blue room, whereas your daughter would be happier in a pink room.

*based on a relatively common (stupid or not) prejustice that gils prefer the color pink over blue and boys prefer blue over pink.

** it is just an example! It may not be sensible beyond its mathematical aspects.

PS: If you have a son and a doughter but only one room, paint it violet... ;-)

Hi Jochen

I was stating that it is a convention (a prejudice) in statistics to deny the possibility of interactions, if the main effects do not exist. I was not stating that such phenomena are not possible. You give an example of one. This convention is a characteristic of being conservative in statistical inference, or rather, of trying to avoid multiple testing. The convention is to favor simplicity, and main effects are more simple than interactions (which can include not just 2-way but 3-way and greater interactions). In general, the level of evidence needed to prove an interaction, is greater than the level of evidence needed to prove a main effect, unless there is prior knowledge favoring the interaction hypothesis. In some disciplines there is a movement toward interactions in favor of main effects. I have colleagues in genomics who believe we have swiped the genome enough to find all main effects for common disorders, and that the focus should be on interactions of genes.

Todd

Ok, Todd, back to this statement:

"I was stating that it is a convention (a prejudice) in statistics to deny the possibility of interactions, if the main effects do not exist."

It this really the case? Possibly I did not get this correctly, but I find this would be quite a stupid convention...

Additionally, it puzzles me because to my knowledge you start analyzing a model including the interaction. If the data does not provide enough evidence for a relevant interaction effect, you simplify the model by removing the interaction, thereby allowing for a simple interpretation of the main effects. But if there is striking evidence for an interaction, you won't simplify the model and you are left with the more difficult task to interpret the results of this complex model. So, you anyway look *first* at the interaction, and only if you can ignore this, you look for main effects... This is just the other way around as said in your statement.

Jochen

Calling my description of what I have experienced as a convention in the literature as "stupid" is a tad unfair.

This is my experience based on many years of interactions with reviewers and editors. I do not object to having interactions in a model, without main effects, if there is compelling evidence to do so, but in the statistics community such models are met with a higher level of skepticism, because implicitly they indicate a multiplicity of tests were carried out.

I certainly do not object to the "backwards" approach to model building that you describe (as in backwards stepwise regression, and not backwards as in wrong), but at what level of interaction do you start? If you have 10 independent variables under consideration do you start with the 10-way interaction, or should only the 45 two-way interactions be under consideration? These are all judgement calls that depend on the application.

I don't think there is right way and wrong way to doing this, as long as each model building exercise is explained and defended appropriately, with regard to the usual aims of making a model fit well (i.e., explain as much variation as possible, while accounting for over fitting bias) and keeping a model interpretable. I do concede that the second aim of interpretability is not important in some fields, such as in the data mining community (the algorithms approach of computer scientists).

Thanks for the discussion

Todd

Thank you very much Todd for your fast response to my question. Your first answer was very helpful for me, stating that indeed it is possible to have main or interaction effects in MR, although the whole regression model is not significant.

I have mentioned main or interaction effects just as an example but was primarily interested in whether a whole regression model can be non-significant but still resulting in significant single predictor (main or interaction) effects.

Nevertheless, concerning your current discussion, in psychology it is at least very common to test both main and interaction effects and we have learnt in our methods classes that it is possible to obtain interaction effects, although there are no main effects. I did not know that this is still not known to other research fields and that in some other research fields, people only test interaction effects when main effects are significant. Very interesting, since we in psychology preceed differently in this issue.