Its good to go with multi group analysis

Hi Muhammad! As you work with a categorical variable you should use multigroup analysis. Moderation is usually used with continuous variables. Also the difference between moderation and multigroup analysis is that moderation analysis is focused only on one relationship between two variables, while with mutigroup analysis the entire structural model is compered between groups. If step 2 of measurement invariance is established you can perform any type of multigroup analysis. When Step 3 is also established, that means that the entire model can be analyzed by using a pooled sample. In PLS-SEM, Permutation multigroup analysis (MGA) is the recommended method. The MGA from the SmartPLS 3 version is recommended only when there is high difference in the sample size of the two groups. See:

Hair, J. F., Sarstedt, M., Ringle, C. M., & Gudergan, S. P. 2024. Advanced Issues in Partial Least Squares Structural Equation Modeling (PLS-SEM) (2e). Thousand Oaks, CA: Sage.

Of course you can apply the three methods implemented in SmartPLS to verify is they give the same results.

It sounds like you used the continuous CA in your standard moderation analysis (SMA), but used the dichotomized CA in your MGA. If so, the difference in results suggests that the moderation effect is not linear.

Linearity of the moderation effect was implicitly assumed in the SMA by using continuous CA; that is, the difference in slopes of the IV-DV relationship was assumed to be the same for each one unit change in CA, across the entire range of scores in the continuous CA. If that assumption is very wrong, a non-significant moderation effect would not be surprising.

In the MGA, you implicitly dropped that linearity assumption in favor of the much weaker assumption that the slope at low-CA will differ from the slope at high-CA, an assumption supported by the significant results in MGA.

You could examine whether the difference in assumptions accounts for your different results by re-running the SMA, but this time using the dichotomized CA instead of the continuous CA. If you get a significant moderation effect from the dichotomized CA in SMA, but a non-significant moderation using the continuous CA in SMA, that would confirm that the different results are due solely to the differing ways of coding CA, and to the differing assumptions about linearity implied by the two different codings.

As to whether SMA or MGA is the more appropriate approach, the key point is made by Vasilica Maria Margalina , namely that SMA "is focused only on one relationship between two variables, while with multigroup analysis the entire structural model is compared between groups." If the MGA reveals substantial between-group differences in parameters other than the moderation effect itself, then the MGA results are more credible than the SMA results for estimating the moderation effect; the SMA fails to control adequately for other effects in the model. If there are no large between-group differences in other parameters, then the MGA results for the moderation effect should be very similar to the SMA results, and the choice of which approach to use comes down to your own preferences and which approach your intended audience is likely to find more familiar.

Dear Burke D. Grandjean ,

Firstly, I apologize, I just now realized that my question was not entirely clear. But yes, you are absolutely right. I used CA as a continuous variable in my SMA analysis, whereas in my MGA analysis, I categorized CA as a dichotomous variable (low vs high).

And thank you for your detailed explanation regarding linearity and moderation. I recall learning at some point that linearity is important in detecting moderation effects, but for some reason, it has slipped my mind. That said, after reading your response, I did some further reading on moderation analysis and I found a source that says "tests of moderation assume that variable distributions include the full range of possible values" (see: https://www.researchgate.net/publication/341357609_PLS-SEM_using_SmartPLS_30_Chapter_13_Assessment_of_Moderation_Analysis)

Does this linearity have anything to do with that assumption? To my understanding, when the variable encompasses the full range of possible values, the data is more likely to be linear. When linearity is assumed, the effect of IV*Moderator on DV would consistently increase or decrease across the entire range. Otherwise, the moderation effect might not manifest evenly and could instead be concentrated at extreme values. Hence, my SMA did not detect a significant moderating effect, whereas dichotomizing the moderator in MGA allowed me to observe the differences more clearly. Would my understanding be somewhat reasonable?

And I also followed your suggestion to test the moderation effect using SMA with dichotomized CA. The result was CA significantly moderate the relationship between IV and the mediator, though it did not moderate the indirect relationship between IV and DV, through the mediator. This is different from when I used CA as a continuous moderator. In that case, CA failed to moderate any of the paths. Absolutely no moderating effect detected. So with this, could I assume that it is a matter of linearity as you explained?

I would be very grateful if you could address my follow-up questions. Regardless, your answer has been very helpful, I truly appreciate it. Thank you again.

Hello Vasilica Maria Margalina !

Thank you for your response! I appreciate the clarification regarding MGA and moderation. However, it appears that I may not have articulated my question clearly and maybe have caused a slight misunderstanding (apologize!)

I did not conduct moderation analysis with CA as a dichotomous variable. Instead, I conducted moderation analysis using CA as a continuous variable, and separately performed MGA using CA as a dichotomous variable (low vs high).

What I was interested in was comparing the results between these two methods, let's call it standard moderation analysis (SMA) vs multi-group analysis (MGA). Because I found that the result differed significantly between these two approaches. The moderation effect of CA was not detected when CA was treated as continuous and analyzed using SMA, but became significant when CA was dichotomized and analyzed using MGA.

Below, you may see Burke D. Grandjean 's answer on the matter along with my response. It seems that the difference in the result stems from an unmet assumption of linearity, which may explain why MGA was able to detect moderation effect in my data and not SMA. However, I'd love to hear your thoughts on this, does this interpretation align with your perspective? Regardless, I truly appreciate your helpful response, and thank you as well for sharing the book recommendation, will certainly look into it!

Dear Jahnavi Munagala

I appreciate your reassuring input, thank you very much.

Hello Muhammad Fahizza Amru ! Is good that you followed Burke D. Grandjean 's advice and perform also a moderation analysis with CA as dichotomous variable. Have you assessed the results of all the MGA tests implemented in SmartPLS (Permutation, bootstrapp, parametric test and Welch-Satterwhaite tests) to see if you obtain the same differences with moderation analysis. Differences can come from the measurement of the variable but also from the statistical methods. Which method did you used to estimate the moderation effect in SmartPLS. It was the two-steps method?

I think linearity and range are distinct issues. I would not necessarily expect a linear moderation effect, in the sense I previously defined for that linearity, to be more likely when the full range is encompassed.

When linearity is assumed, the effect of IV on DV is forced by the model to consistently increase or decrease across the entire range of the Moderator in the sample data, whether or not that covers the entire range in the population, and whether or not the moderation effect is in fact linear in the data. If it's not linear in the data, but is modeled as linear, it will generally be estimated as weak and non-significant by the model. If the effect, though non-linear, is on average a lot different at the upper end of the CA range in the data from what it is, on average, at the lower end, then the dichotomous CA could be strong and significant.

Yes, your results comparing SMA with continuous CA to SMA with dichotomous CA do suggest that the difference is due to non-linearity of the moderation effect. And, to repeat: If the MGA reveals substantial between-group differences in parameters other than the moderation effect itself, then the MGA results are more credible than the SMA results for estimating the moderation effect; in that case, the SMA fails to control adequately for other (group-specific) effects in the model. If there are no large between-group differences in other parameters, then the MGA results for the moderation effect should be very similar to the SMA results, and the choice of which approach to use comes down to your own preferences and which approach your intended audience is likely to find more familiar.

Muhammad Fahizza Amru I also have to add that you must to take care when you choose the settings of the boostrapping method because some MGA tests are one-tailed and others are two-tailed. The permutation method is two-tailed, so the in bootstrapping settings for the moderation results you must choose two-tailed in test type. On the contrary, the MGA from SmartPLS is a parametric test and one-tailed. This means that when applying boostrapping for testing the significance of the moderation effect you must choose a one-tailed test. If not the conclusions when you compare the results may not be accurate.