Typically, when bootstrapping CI does not include zero, the indirect effect is considered significant. But does it matter if the lower bound is really near to zero (00.2)? Does that somehow "weaken" the significance? How can we interpret such results?
I will recommend that you read Bootstrap confidence intervals Class 24, 18.05 Jeremy Orloff and Jonathan Bloom .
You can get it at the link below
https://ocw.mit.edu/courses/mathematics/18-05-introduction-to-probability-and-statistics-spring-2014/readings/MIT18_05S14_Reading24.pdf
If you are viewing things through the lens of significance, then you are drawing a yes / no conclusion. Not containing zero is not containing zero. That's the end of the story for significance. Of course, the beauty of using something like confidence intervals rather than, say, a significance test, is that you are able to apply some judgement. You can assess a confidence interval that is close to zero as something different than a confidence interval that is far from zero. You can report these values or plot them, so that your audience can make their own conclusions as well. When someone claims, "I'm not broke!" and as evidence produces a bank statement of $0.02.... well, they're right, but we might not be too impressed with their wealth...
We cannot change the rules just because the lower bound is 0.002.
What is the upper one? If it is 0.004, then where is the problem?
Could you please give us a typical CI of your results?
If the effect is negative and both upper and lower boostraps are negative thus do not include 0, is that significant? Thanks
Shelley Swift , yes, that's a negative effect size statistic whose confidence interval does not include 0. So, you can see that as "significantly" different from a 0 effect size, if that's the question you are asking of the statistic... Some effect size statistics are signed, in the sense that they can be positive or negative. For example, r (the correlation coefficient), and Cohen's d and Cliff's delta. When there are two groups (Cohen's d, Cliff's delta), a negative effect size statistic (should) imply that the second group is larger in (mean, median, whatever) value than the first group. This is analogous to a negative t value in a t-test. And we are familiar with how to interpret a positive or negative r value. There are also effect size statistics that aren't signed. That is, they are always positive. This is analogous to an F value in an F-test. An example of this for effect size statistics is Cramer's V. Interpreting the confidence interval for this kind of statistic can be tricky, depending on how the confidence interval is calculated. I'll add an additional response for this case.
Another consideration in the case that the OP brings up is that if the effect size is one that CANNOT be negative, and the confidence interval is calculated in certain ways (for example, with a percentile bootstrap method), then the confidence interval CANNOT cross zero, and therefore can't be used to determine the "significance" of the relationship. In this case, really, a significance test should be used (like the traditional test, e.g. chi-square test of association, F-test in anova, or whatever), and the point estimate and confidence interval of the effect size statistic can still be presented, but the reader should be reminded that this interval can never cross zero.
- Badges
- Science topic
- Similar topics
- Biological Science
- Physiology
- Physiological Processes
- Stress