Dear colleagues, I would deeply appreciate your feedback on the following workflow. It is a moderation analysis, in which we are interested in whether the relationship between X and Y changes across levels of a) a grouping factor with 2 levels, b) a continuous variable M1 , and c) a continuous variable M2 .
We could have used a multiple regression model in order to include all variables, and we actually did: the four-way interaction is significant. However, in order to keep the interpretation simple, we broke down th problem in two parts.
Thus, we examine the relationship between X and Y, across the quantiles of M1 and the two levels of G, and then we examine the relationship between X and Y, across the quantiles of M2 and the two levels of G.
We report the results in a table like this:
Quantiles | beta of Y ~ X for (blanks are not-significant betas)
of M1 | Level 1 of G Level2 of G
10 | -0.3
25 | -0.2 -0.2
50 | -0.3
75 | -0.35
90 | -0.4
of M2 | Level 1 of G Level2 of G
10 | -0.8 -0.8
25 | -0.28 -0.3
50 | -0.01
75 | -0.4
90 | -0.6
The pattern of the results is indeed interesting and it justifies using a moderation analysis, since it shows that there is a difference in the relationship across levels of all the moderator variables.
Is it acceptable to deal with the two continuous moderators separately? What if we add a Y2? Are we inflating the Type I Error probability too much, due to performing four tests instead of one?
I would appreciate any feedback on these questions. Thanks in advance!
Hello there. I've quite a bit of experience with analyses for moderation so I might be able to help. My first thought after reading was, "what are their research questions?" To me, these will dictate the appropriateness of the analyses that are conducted afterwards. If you have put forward a very specific triple-moderator question/hypothesis then I really don't see how you can avoid running the complete interaction (with the issue then becoming how best to conduct and present this to tell a 'story' that is easily interpreted by readers and reviewers).
From you first paragraph, it looks like you may have separate research questions for each of your three proposed moderators. In that case, you might like to build your analyses up from 2-way interactions. How does the 'story' build-up? If you find evidence of a contextualising effect, is that itself contextualised by another? (leaving issues of moderated-moderation [etc] aside).
At the moment however, it looks like you've cherry picked only part of a story - by conducting two separate moderation analyses (M1*G and M2*G) and then qualitatively comparing them by presenting them next to one another. This really doesn't seem appropriate to me.
Yes, there are a number of alternative approaches you might use to the analyses, but the appropriateness and availability of these depend on the statistical software packages that you have available (e.g. latent class analysis in Mplus). The biggest issue by far though is: What are your research questions? To propose a triple moderation as justified from the literature would require working in a really very mature field. I'm happy to help further.
Thanks a lot for your feedback. Your reply clarified a couple of things for me.
The question was "whether M1 and/or M2 moderate the Y~X relationship". I then introduced the G factor, due to the matrix of partial correlations showing different relationships between the variables for G1 and G2 .
The tables weren't supposed to allow comparison of the relationship across quantiles of M1 and M2, since the M1 * M2 term was not calculated. I see how this presentation is misleading.
Regarding available methods, we can also use SEM or LCA in lavaan and Mclust in R. Do you think it is better to approach this question with these techniques?
For the time being I will try to both build up the argument from the two way interactions, and break down the significant four-way interaction.
Thanks again for taking some time to review my question!
No worries, my pleasure to help. It sounds like you need to build-up a story to a 3 way interaction between X M1 and M2. Testing X*M1 and X*M2 first, and then looking at X*M1*M2. The G categorical moderator seems like it needs to be a statistical control only. You might like to regress X, M1, M2, and Y on G before hand, saving the residuals out as one means of controlling for the effect of G (akin to ANCOVA), but that's a bit clunky. An easier way would just be to put this into a SEM programme and have G as a covariate predicting Y and appropriately covarying with X, M1 and M2.
The LCA approach might be an interesting use of time to succinctly approach this issue though. The issue would be, "Are there distinct patterns in the association between X and Y?", "Do M1 and M2 predict these distinct patterns, and if so how [this leads to adding in an interaction between M1 and M2]?" You could always vary the second of the questions to add in G and so end up presenting a 3 way interaction of M1, M2, and G predicting a (hopefully for the sake of simplicity) binary outcome measure that captures variation in the X-Y relationship (I'm not sure this is necessary more than adding G as a covariate though). This approach would seem like a fun and interesting way to explore the issue, as well as one that might be easier to write up. :-)
LCA sounds interesting indeed.
I actually tried to approach the problem with prediction of categorical outcomes (a have posted a separate question on that in the attached link). But in that case I combined high levels of Y1 and Y2, due to the distributions being skewed and the high scores being of clinical importance.
I will implement both your suggested approaches, especially looking forward to examining the problem with LCA. I will definitely learn something...
Thanks again for your feedback!
https://www.researchgate.net/post/Expectedly_Skew_Dependent_Splitting_at_08_quantile_then_performing_MANOVA_and_separate_LDAs_for_two_subsets_of_the_sample?_tpcectx=qa_overview_asked&_trid=dpBEkvo8KNnqIHPVHJBZlT7h_
- Badges
- Science topic