I've often seen the following steps used to test for moderating effects in a number of papers, and I don't quite understand the usefulness of Model 1 (which only tests the effect of the control variable on the dependent variable) and Model 4 (which only adds the cross-multiplier term of one of the moderating variables with the independent variable). These two models seem redundant.
The effects of the control variables in Model 1 might be changed substantially by introducing the additional predictors in Model 2, and/or the moderating effect of the first moderator alone, shown in Model 3, could be changed substantially when controlling for the second in Model 4. In your results neither of those things happens, but they could have happened. Reporting all four models does not take up a lot more space, and allows the reader to check for that. If they did happen, it might indicate multicollinearity, or there could be important theoretical implications.
Burke D. Grandjean Thank you for your reply, but I have a few disagreements. First, good control variables should be those that are common causes of the independent and dependent variables, that is, XY. At this point, the effect of the control variables in model 2 should change relative to model 1, because we are blocking the effect of C on a portion of Y, that is, C-->X-->Y. Thus, the coefficients on the control variables don't change much between models 1 and 2, which may indicate that the confounding problem is not serious, or it could be that our control variables are not good enough (e.g., C will only have an effect on Y alone or not at all). The existence of Model 1 does not tell us any additional information. Second, in response to your mention that it can reflect multicollinearity, I don't think the existence of model 1 helps. This is because it is not clear whether the change in the coefficients is a problem of causal identification or a problem of statistical inference. Overall, I feel that Model 1 is redundant when it exists, does not give us any additional information, and may even mislead us.
As you say, "the effect of the control variables in model 2 should change relative to model 1, because we are blocking the effect of C on a portion of Y, that is, C-->X-->Y. Thus, the coefficients on the control variables don't change much between models 1 and 2, which may indicate that the confounding problem is not serious, or it could be that our control variables are not good enough." That was precisely my point: unless you report Model 1, the reader has no way to know whether the coefficients in Model 2 do or do not change much "relative to Model 1." In other words, the reader can't make a judgment about how the two models compare unless you report both models. If the coefficients had changed much (which they didn't, in your example), then multicollinearity would have been one of the possible explanations to be considered, along with the causal-structure issues you mention.
Burke D. Grandjean Thank you very much for your patience! I agree with you, though I still don't think there is much value in performing this manipulation in the results section.
If the difference between the two coefficients is not large, I cannot be sure if this is because the risk of confounding is low in my study or because my control variables are null (not eliminating the risk of confounding). Conversely, if the difference between the two coefficients is large, again I cannot determine whether this is because I am controlling for confounding well or because of some other problem (such as the multicollinearity problem you describe).
What is the point of this manipulation, given my difficulty in determining the reason for a particular situation? I think the relevant discussion should focus on the relationship of the control variables to the independent and dependent variables, with a dedicated section for detailed discussion and further testing.
- Badges
- Science topic
- Similar topics
- Social Science
- Media
- Journalism