Dear colleagues,
I have carried out a multiple mediation (model 6) with PROCESS, and I have got the next results:
Indirect effect(s) of X on Y:
Effect BootSE BootLLCI BootULCI
TOTAL ,0155 ,0021 ,0116 ,0198
Ind1 ,0078 ,0013 ,0054 ,0106
Ind2 ,0075 ,0016 ,0045 ,0106
Ind3 ,0002 ,0001 ,0000 ,0004
As far as I know, the CI must not contain Zero. Taking into account that Bootstrapp LLCI of Ind3 = .0000 (PROCESS does not show more decimals so it could be .0000003), is there no effect in case or should LLCI be negative to consider there is no effect?
Thank you very much for your replies.
Kind regards,
Gonzalo
First, you can ask process to provide more decimals, I think. If zero is not included in the CI intervall, it seems your effect is significant. However, it seems very very ... very very and i can not stress this enough, very small. I would take into account, what this effect means. For example, if these are unstandardized regression coefficents, and lets say your scale is from 1 to 7. Is an indirect effect of a .0002 or smaller practically meaningful to you? Also, bootstrapping is based on taking many samples from your sample. Therefore, the estimates of the CI can vary with each estimation. Did you try again (or with another seed?)? Does ist replicate the significant CI? I recognized that if effects are rather small it can vary whether zero is included in the CI, thus your effect, because so small, is not very reliable. I would not put to much trust in it (besides considerations of practical relevance). Also, did you try heteroscedasiticity consistent standard errors? What hapens when you use, lets say, the HC3? I would be very sceptical of these small indirect effects (all three of them by the way) - but of course this depends on what .002 practically means.
I would like to add to Franziska's answer a question (besides the concerns regarding the effect size/size of the indirect effect itself): would you come to a different conclusion and interpret your results differently, if the lower bound of the bootstrap CI would be 0.0003 vs. 0.0001 vs -0.0001? This is all within the reasonable variation from bootstrap sample to bootstrap sample I would guess.
@rainer the answer is clearly YES :) because remember "we only care if the interval does not cover zero" (I am obviously being ironic).
@gonzalo You need to learn more about the tools you are using
For instance you should know how those intervals are computed or what coverage they have (90%,95%,99%, 3%?), otherwise you are just throwing numbers at people and it is kind of pointless. You can find these answers googling or looking at the manual of your software.
You can actually compute those bounds manually and you would know that the lower bound is 4e-06, while the upper is 4e-04.
You also say "As far as I know, the CI must not contain Zero", says who?
You should use Confidence Interval to check how much variation you have in your estimates, nothing more.
Although I agree with both Franziska Ehrke and Rainer Duesing , the problem is far beyond the random nature of resampling, or its seed [changing the seed would make sense if the initial conditions were different for every seed, but the bootstrap converges at some point, If the results change dramatically, that means that those two bootstrap runs need to be merged together and possibly some more added. One should always use the maximum number allowed by the machine/good reason.
So the actual question is: How many iterations did you use? again we do not know]
Trying different seeds to me makes little sense, and creates even more confusion.
We should rather a bootstrap approach that can handle specific characteristics of the data (heteroskedasticity for instance)
That effect is actually significant at 5% (I despise this word): the corresponding pvalue is 0.045, but again is an effect of 0.0002 relevant?
The whole point of using the bootstrap is that the sampling distribution could be nonnormal, but the output clearly uses standard errors, which makes the whole thing pointless
Thank you very much for your explanations and for your time, Stefano Nembrini .
Questions are never impertinent., but sometimes answers tend to be ("you need to learn more", "you are just throwing numbers", "I despise"). I did not reflected all the data and did not cite any author because I was writting to colleagues, I was not defending a doctoral dissertation and I preffered to be brief.
My confidence level for confidence intervals was 95%, and I used 10,000 boostrap samples. On the other hand, Hayes stated that if zero is outside of a CI% confidence interval, there is evidence that X affects Y through M (Hayes, 2013).
Kind regards,
Gonzalo
Just because you can find a reference does not mean that is correct. This works if you design your experiment with error i and ii that are defined. Otherwise the confidence interval trick makes no sense: you can generate at least 10 intervals from a single bootstrap ,you have a high chance of finding one that excludes 0.You should focus on those comments that suggested that the effect is too small to be of interest
- Badges
- Science topic
- Similar topics
- Mathematical Sciences
- Statistics
- Statistical Software
- SPSS