There are two ways we can go about testing the moderating effect of a variable (assuming the moderating variable is a dummy variable). One is to add an interaction term to the regression equation, Y=b0+b1*D+b2*M+b3*D*M+u, to test whether the coefficient of the interaction term is significant; an alternative approach could also be to equate the interaction term model to a grouped regression (assuming that the moderating variables are dummy variables), which has the advantage of directly showing the causal effects of the two groups. However, we still need to test the statistical significance of the estimated D*M coefficients of the interaction terms by means of an interaction term model. Such tests are always necessary because between-group heterogeneity cannot be resorted to intuitive judgement.
One of the technical details is that if the group regression model includes control variables, the corresponding interaction term model must include all the interaction terms between the control variables and the moderator variables in order to ensure the equivalence of the two estimates.
If in equation Y=b0+b1*D+b2*M+b3*D*M+u I do not add the cross-multiplication terms of the moderator and control variables, but only the control variables alone, is the estimate of the coefficient on the interaction term still accurate at this point? At this point, can b1 still be interpreted as the average effect of D on Y when M = 0?
In other words, when I want to test the moderating effect of M in the causal effect of D on Y, should I use Y=b0+b1*D+b2*M+b3*D*M+b4*C+u or should I use Y=b0+b1*D+b2*M+b3*D*M+b4*C+b5*M*C+u?
Reference: 江艇.因果推断经验研究中的中介效应与调节效应[J].中国工业经济,2022(05):100-120.DOI:10.19581/j.cnki.ciejournal.2022.05.005.
First of all, if you include additional predictors in the model, it is expected that the regression coefficient of your focal variable (here D) will also change, but this is beyond the point of your question.
In my opinion, the inclusion of C and/or C*M does not change the general interpretation of D. If you rearrange the equation, you can see that the influence of D depends on b1 and b3*M --> Y=(b1+b3*M)*D+b0+b2*M+b4*C+b5*M*C+u
Therefore, the inclusion should depend on your hypothesis, how the control variables influence Y. Maybe a directed acyclic graph (DAG) may help to make it more clear.
Rainer Duesing Thank you for your answer. I would like to discuss this issue with you further. First, it has been made clear whether the addition of M*C to the interaction term model Y=b0+b1*D+b2*M+b3*D*M+b4*C+u will not change the interpretation of the coefficients of D. However, the addition of M*C will change the value of the coefficients of D. The decision to include M*C or not is based on my assumptions. Second, in Y=b0+b1*D+b2*M+b3*D*M+b4*C+u, I have assumed, equivalently, that the moderating variable will not change the effect of the control variable C on Y. Instead, in the grouped regression model with the control variables added, the control variables have been grouped according to the value of M. Therefore, grouped regression is equivalent to defaulting to the assumption that "the moderator variable affects the effect of the control variable C on Y".
In summary, if I believe that the moderator variable does not affect the effect of the control variable on the dependent variable, we should use a regression model that does not include the cross-multiplication terms of the moderator and control variables, that is, Y=b0+b1*D+b2*M+b3*D*M+b4*C+u. When we use grouped regression, it is equivalent to Y=b0+b1*D+b2*M+b3*D*M+b4 *C+b5*M*C+u. In this case, it is equivalent to accepting the hypothesis that "the moderator variable affects the effect of the control variable C on Y".
I think it is fine to leave out thw M*C interaction if you believe that this will have no additional value. But since you are not interested in C but in D AND you want to improve the predictive value of D, it might be useful to check, if the M*C interaction adds anything. I would recommend not to rely on change in R^2, but to have a look at information criteria like AIC and BIC, which also penalize model complexity. Additionally, information criteria can also get worse as compared to R^2, which can only be >= 0.
Rainer Duesing Thank you for your reply. I would like to explain once again my concerns on this issue.
In fact, my main concern is how to test the moderating effect of M. When testing the moderating effect using grouped regression, we need to test the significance of the coefficients of the interaction term -M*D - after performing the grouped regression. At this point, if I use Y=b0+b1*D+b2*M+b3*D*M+b4*C+u (1), the coefficient of D (when M is a dummy variable, at which point the coefficient of D can be interpreted as the average causal effect of D on Y when M=0) is inconsistent with the estimate in the grouped regression (when regressing the group with M=0 alone). When the M*C term is added, that is, Y=b0+b1*D+b2*M+b3*D*M+b4 *C+b5*M*C+u (2). the coefficient on D is only consistent with the estimates in the grouped regression. So, my question is, if I were to test the moderating effect of M in a grouped regression, should I focus on the coefficients of D*M in equation (1), or should I focus on the coefficients of D*M in equation (2)? Which equation has the correct coefficient on the interaction term (i.e., b3)? As I discussed above, I think this depends on our assumptions.
Maybe we are not on the same page here. Can you please elaborate what you specifically mean by "grouped regression"? I thought that you mean the moderation effect of M as grouping factor. What else did you do or make it clearer. I am a bit confused. Did you calculate several regression models for the M=0 and M=1 groups separately? If "yes", why? Maybe it would be helpful to present the outputs.
Rainer Duesing Apologies for not expressing this clearly. Briefly, yes, I started with separate regressions for the M=0 and M=1 groups, which has the advantage of directly showing the causal effects of these two groups. However, grouped regressions cannot judge intuitively the between-group heterogeneity between these two groups. For example, the causal effect of D on Y is insignificant in the M=0 group and significant in the M=1 group, but the difference between them is likely to be insignificant. Therefore, we need to further test the between-group heterogeneity of the causal effect of D on Y by means of an interaction term model, which is expressed as the statistical significance of the coefficient estimate of the interaction term D*M.
Everything you wrote in the last comment is clear to me, although I still do not know why you are conducting a subgroup analysis for M=0 and M=1 separately, since it will have less power (with equal sample size, half of the degrees of freedom) and you will have no additonal information, quite the opposite, you are lacking the test for the difference between the slopes, i.e. the interaction term.
Back to your last question: I cannot see that the general effect of D will change when you include C. Of course it matters if you conduct subgroup analyses or moderation, but the general pattern should remain the same. What may happen and alter the results, if D and C are correlated or even more complicated, if they a a (mutual) suppression effect. Therefore it is mandatory to look at the bivariate correlations.
As requested above, please share your outputs/results from your correlations and your moderation as well as the subgroup analyses. Maybe we can see your problem better then. This is very theoretical at the moment and very speculative.
Rainer Duesing As shown in the figure, the coefficients of D in the grouping regression (models 3 and 4) are consistent with the results in model 2. Specifically, when M=0 (Model 3), the coefficient of D is 0.005, and when M=1 (model 4), the coefficient of D is 0.02. The same is true in model 2, when M=0, the coefficient of D is 0.005, when M=1, the coefficient of D is 0.005+0.014, which is about 0.02. This is quite different from the coefficient of D in Model 1. In model 1, when M=0, the coefficient of D is -0.059, when M=1, the coefficient of D is -0.059+0.089, which is about 0.03. Similarly, the coefficient of D*M, which we focus on to prove the significant moderating effect, also has a significant difference, with the value of this coefficient in model 1 and model 2 being 0.089 and 0.014, respectively. So, if we want to test the moderating effect of M in D on Y, should we use model 1 or 2? So far, almost all existing studies have used model 1.
Thank you for the outputs, this makes it clearer and I can see the cause. If I remember correctly, your a priori premise was that there is no C*M interaction, but as you can clearly see in model 2, there is one. This is the cause why the subgroup analysis for M=0 reveals the same parameter estimates as model 2 and not model 1. By using subgroup analysis for M, you not only test a moderation with D (what you originally wanted) but also the interaction with C!!
As you can see, the changes of the parameter values for D and C from model 3 to model 4 are equal to the values of the interaction terms of model 2, respectively. D[M==0] = 0.005, D[M==1] = 0.02; 0.02-0.005 = 0.14 (rounding error) = b3 model2 for D*M. Likewise, C[M==0] = 0.023, C[M==1] = 0.157; 0.157-0.023 = 0.135 (rounding error) = b5 model2 for C*M.
Therefore, you can only expect that the estimate for D[M==0] is equal to D in model 1, if the interaction M*C is zero, which is not the case. Model 2 allows an intercation and this is also represented in the subgroup analyses.
Hence, as I poposed earlier, it makes sense to incorporate the interaction M*C, although you do not have a hypothesis for it and back it up with information criteria.
A final comment: I dont know what your variables represent, but the effects look rather small and you should think if this is of any practical relevance.
P.S.: the sample size(s) and the residual standard deviation look so neat, are this simulated data or real data?
Doubts about what? The algebra is unambiguous here. If there is a non-zero C*M interaction, then the subgroup analyses for M also implicitly includes the C*M interaction and not only the D*M interaction. That is fact. If the C*M interaction would be zero, the estimates for D in model 1, 2 and 3 would be the same. Since you have a non-zero C*M, this is not the case.
The problem is not in the model, but maybe in previous research which missed this? Your sample size is very large and the effect of C*M comparably big, so that this does not look like a random fluke. It may be a bias in your sample, but this is a whole other issue and cant be solved within this model.
- Badges
- Science topic
- Similar topics
- Social Science
- Media
- Journalism