Hi dear researchers,
I need your guidance on the moderation test of a reflective construct (i.e., Uncertainty avoidance with three items) with high and low UA.
I run the moderation test with the interaction effect and found it non-significant.
After that, I also run the moderation by splitting high and low values through multi-group analysis and found it significant for both groups (high and low UA). Now, here I am confused what should I follow?
Need your guidance, please advise. thank you
@Asif Ali Safeer
Moderation and Multi-Group Analysis are quite different from each other. Moderation refers to buffer or enhancement effect of a moderator on an existing relationship between predictor and outcome variables. Whereas, Multi-Group analysis is meant for looking whether effect size for hypothesized paths are invariant/variant by segregating a sample into two or more sub-samples based on a given factor (be it gender, age etc. In your case, you need to check whether paths are significant variant for low and high UA or not using critical difference ratios. If yes then it is moderating significantly else it is not.
Hello Asif,
my thoughts/questions
1) You shouldn't split metric variables into groups as you loose information and reduce power dramatically
Irwin, J., & McClelland, G. H. (2001). Misleading heuristics and moderated multiple regressions models. Journal of Marketing Research, 38(1), 100-109.
Dimitruk, P., Schermelleh-Engel, K., Kelava, A., & Moosbrugger, H. (2007). Challenges in nonlinear structural equation modeling. Methodology, 3(3), 100-114.
Maxwell, S. E., & Delaney, H. (1993). Bivariate median splits and spurious statistical significance. Psychological Bulletin, 113(1), 181-190.
These papers clarify that beyond loosing power you can also find spurious moderator/interaction effects.
2) In contrast, the moderated regression analysis (with the product term) will be misleading, if the interaction is nonlinear--e.g., if across the wide range of Z, the X-Y relationship is constant and then suddenly goes up. This will be undetected with a moderated regression as it tests the linear increase of the X-Y-relationship across Z. In this case, a generalized additive model (GAM) would be an effect, powerful and easy way to do (in R).
Finch, W. H., & Finch, M. H. (2018). A simulation study evaluating the generalized additive model for assessing intervention effects with small samples. The Journal of Experimental Education, 86(4), 652-670.
A last way regarding concept: Moderation is a statistical concept and means the same as interaction. Moderated regression and multi-group modelling are procedures to approach them. They are both equivalent. For instance, a dummy moderator D can be modeled by a X*D product term in a moderated regression or by a multigroup model with both D groups modeled as groups.
A moderator is no causal concept. It just means that a relationship (that may stem from a variety of reasons) varies across levels of a third variables. As moderator hypotheses are in 99% of the cases causal hypotheses, thus, using the term "effect modifiers" would be better to express the beliefs of the researcher that the modifier is supposed the alter the causal effect of the X variable. This in turn would demand some effort to causally identify the effect by using instrumental variables or appropriate control variables.
VanderWeele, T. J. (2009). On the distinction between interaction and effect modification. Epidemiology, 20(6), 863-871.
HTH
Holger
Dear Dr. Holger Steinmetz , Thank you for your detailed response. Can I use MG Analysis for moderation to check the high uncertainty avoidance between the high and low consumer groups? Need your guidance on this point. Thank you
Dear Imran Anwar Thanks for your response. I have seen many high IF papers where the authors used the moderator through multigroup analysis to check the high and low group significance. It would be great if you kindly share any reference on your point for further clarification. Also great to see your guidance on my question. thank you
Actually, I responded to that with the first note ("don't split metric variables").
I would
a) do some graphical analysis to inspect distributions of all variables, and scatterplots (perhaps 3d)
b) estimate a GAM. That means to go out of your comfort zone, download R, search for some code how to import a csv file (which can be done with read_csv2() from the readr package and estimate a GAM (whose code is in the paper that I linked).
If you came so far that you managed to import the data, come here again and I help you with the rest.
All the best,
--Holger
Dear Dr. Holger Steinmetz Thanks for your quick response. I understand your point of view but I am not an expert in other software except AMOS/SMART PLS. So, it would be great if you kindly guide me considering my expertise limitations. thank you so much
- Badges
- Science topic
- Similar topics
- Counseling Psychology
- Confusion