Did you centre the moderator and the IV? The effect of an IV in a model with a moderator is typically changed by the inclusion of a moderator because these are different effects conceptually.

Hello Thom Baguley and thank you for your feedback! IV and DV are both binary variables that I did not mean centre. Do I understand your answer right, that a direct effect between IV and DV can be different only based on whether the model includes a moderator or not, even if for the direct effect the influence of the moderator is not considered (I am only considering the direct effect, not the interaction effect)?

Yes - if you refit the moderation models with centred predictors you should get more consistent results. The moderation model evaluates the IV effect when all variables are zero; this is an issue because the IV effect varies according to the moderator so you're evaluating it at a point where it is often meaningless or there is very little information. (It can get more complicated if there are other predictors in the model that are correlated with IV or moderator but that's likely the crux of the issue). The interaction model needs to be interpreted more holistically than a main effects model because the effects aren't independent so looking at single parameters will often be very confusing.

Dear Thom Baguley this is very helpful, thank you loads! I indeed got different results for when my predictors are mean centered. Do you happen to have a source that states your saying from above that "The moderation model evaluates the IV effect when all variables are zero"? I would like to quote this in my thesis to highlight why different effects are observed.

I could be cheeky and just say: the definition of an interaction and intercept in a regression model.

You can quote my book (Serious stats - there is a chapter on interactions/moderation) or that of a number of other advanced regression texts (Aiken & West, Cohen et al., etc.)

The crux is that you have a main effects model:

y = b0 + b1X1 + b2X

Here b1 and b2 are the effects of the predictors (you call them IVs but strictly that's just if its a designed experiment). They are evaluated when all other predictors are zero, but as the effect of b1 and b2 are constant additive effects they are the same (by definition) regardless of the value the other variable (X1 or X2) takes.

In the moderation model:

y = b0 + b1X1 + b2X + b3X3 (where X3 = X1 * X2)

the same principle applies except now the evaluation of b1 when X2 is zero matters a lot (in most cases). This is because the variables are correlated so this causes the intercept to shift in most cases. We can restore the intercept by centering X1 and X2 prior to computing X3. This makes the intercept the mean and we're evaluating the effects of X1 and X2 then they are both at their means (now zero) which should be essentially identical to the values in the main effects model. This can all happen even if the interaction is non-significant (and will do barring some rare circumstances).

Note that centering doesn't actually change the interpretation of the model - just the individual coefficients. If we say get predictions from the model everything works out just fine (and indeed one of the best ways to interpret these models is to plot the predictions of X1 as a function of X2 or vice versa).

Aiken, L. S., & West, S. G. (1991). Multiple Regression: Testing and interpreting interactions. Thousand Oaks: Sage.

Baguley, T. (2012). Serious stats: a guide to advanced statistics for the behavioral sciences. Palgrave Macmillan.*

Cohen, J., Cohen, P., West, S. G., & Aiken, L. S. (2003). Applied multiple regression/correlation analysis for the behavioral sciences, 3rd ed.. Hillsdale: Erlbaum

* All these books will cover the core ideas here, though I like my own book. You may have access to it from your university library (if not I'd recommend asking them to order a few copies!). Aiken and West is a very good book focused just on interactions in regression. Cohen et al. is wonderful advanced book on regression. My book has less in depth on regression but probably more breadth on that and some more general stats topics.

To clarify further: ""The moderation model evaluates the IV effect when all variables are zero". All regression models do this - they evaluate the effects of X when all other Xs are zero. It just doesn't matter in a model without interactions because the effect of X1 when say X2 is zero is the same as when X2 is not zero) because its a linear, additive model. Y is always just a prediction from addition up b1 * X1 + b2 *X2 etc. Its only an issue in a non-additive model.* An interaction model is inherently non-additive - its designed to detect non-addivity in the effects of X1 and X2.

This also arises in certain other models notably polynomial models (but they're really also just interaction models.

* Its a little sneaky because it does this by estimating the additive effect of X3 (where X1*X2 = X3)

Thom Baguley thank you very very much, I get it now. Very helpful explanations and sources!