Example from my analysis:
A and B (main predictors) predicting C (mediator) and C predicting D (outcome variable).
Example 1 : Analysis without bootstrapping:
Direct effects:
A - D = positive, sig.
B - D = negative, non-sig.
Indirect effects:
A - C = positive, sig.
B - C = negative, non-sig.
C - D = negative, sig.
Total indirect:
A - B - D = negative, sig.
A - C - D = non sig.
Example 2: with 1000 bootstraps
Direct effects:
A - D = positive, non-sig.
B - D = negative, non-sig.
Indirect effects:
A - C = positive, non-sig.
B - C = negative, non-sig.
C - D = negative, sig.
Total indirect:
A - B - D = negative, sig. (based on 95% confidence intervals, not on p value).
A - C - D = non sig.
Although in both examples, total indirect effects are congruent, I have serious doubts:
1. Why is it that some paths are significant without and become non-significant with bootstrapping is added? (Estimates [coeffiecients] do not change, but p values do)?
2. In example 1, total indirect effect and partial indirect effects are significant, despite the fact that the direct effect is still significant. Is that not weird? In that case, shoudl not the direct effect be non-significant (A-D)?
In conclusion, what method is superior? Should I just always use bootstaps if the sample is representative?
Bootstrapping is typically used because the indirect effect is itself often nonnormal, which invalidated analytically-derived SEs and thus p-values and their corresponding 95% CIs. Bootstrap CIs have become the standard solution. However, Monte Carlo CIs can work reasonably well and the 95% intervals from a Bayesian analysis done with MCMC are also fine for nonnormal effects.
Hi Witold, you already got valuable advice by David an Solomon.
Here are my two cents:
"1. Why is it that some paths are significant without and become non-significant with bootstrapping is added? (Estimates [coefficients] do not change, but p values do)?"
-> If p-values change, this can only be due to different estimated standard errors (SE). Bootstrapping takes non-normality of the data into account, while SEs resulting from maximum likelihood estimation are based on the normality assumption.
"2. In example 1, total indirect effect and partial indirect effects are significant, despite the fact that the direct effect is still significant. Is that not weird? In that case, should not the direct effect be non-significant (A-D)?"
-> No, not necessarily. If the direct effect is still present after the mediator variable was added to the model, then the mediating variable accounts only for some, but not all of the relationship between predictor and criterion variable. This is called partial mediation as opposed to complete mediation, when the predictor variable no longer affects the criterion after the mediator variable has been controlled. Partial mediation is the case in which the path from predictor to criterion is reduced in absolute size but is still different from zero when the mediator is introduced.
"In conclusion, what method is superior? Should I just always use bootstrap if the sample is representative?"
-> If your sample size is large enough, bootstrapping should generally be superior.
Kind regards, Karin
Hi Karin,
Referring to your answer of first question, Can u explain more? I mean, is it a problem that some direct paths are significant without and become non-significant with bootstrapping is added? How can I explain it?
Hi Mahiuddin, David Boyda and Solomon Kurz already provided some answers concerning bootstrapping. Bootstrapping is often used for estimating standard errors of the direct and indirect effects. On the other hand, maximum likelihood (ML)also provides standard errors. Both estimations may be different depending in the distribution of the variables and sample size. Therefore, the method for estimating standard errors should be justified.
Therefore, if a direct path coefficient is significant using ML and this coefficient is non-significant using bootstrapping, then you should investigate the assumptions necessary for using each of the methods.
Kind regards, Karin
Bilal Mustafa Khan
Bootstrapping may be a good alternative, but it is just one method out of a plethora of methods. For nonnormally distributed data, you could also use robust maximum likelihood (MLR) or several weighted or unweighted least-squares methods.
By the way, there are different bootstrap methods available such as the percentile method or the bias-corrected and accelerated approach. For each estimation method used and for each bootstrap method used you will need to provide a justification.
Kind regards, Karin
Hi Karin & everyone,
Thank you for the previous discussion. I am using WLSMV and I am getting different results results depending on whether or not I am using bootstrapping in addition. Do you have any resources to share or any thoughts on WLSMV combined with boostrapping? I am wondering if WLSMV is taking account non-normality sufficiently in my sample (n = 1,027) so that I don't actually need boostrapping?
- Badges
- Science method
- Similar topics
- Mathematics
- Statistics
- Statistical Techniques
- Structural Equation Modeling