Hi everybody,
differently than in meta-analyses, the effect of publication bias in meta-regression seems to me less severe for the slope coefficient,
In my opinion, a bias in the slope coefficient may result only if the moderator levels are correlated with the probability of being published. Otherwise, the result is just a loss of statistical power (i.e. higher confidence interval).
However I found a paper where a simulation is run telling that also slope is affected no matter what (see the link).
Have I misunderstood some part of the simulation? Or instead, is my opinion wrong?
Thank you!
Massimiliano
http://www.ncbi.nlm.nih.gov/books/NBK84231/pdf/Bookshelf_NBK84231.pdf
I suspect that publication bias certainly affects regression slope coefficients, if those are the effect that is under study. A slope coefficient is an effect size, and publication bias is known to inflate effect sizes in general.
I agree with Stephen: "publication bias" means that studies do not have equal probabilities of being published if their results are lying in a certain 'direction' of the effect rather than the other. Thus, you are loosing more information on one side rather than the other, and your estimated effect size will be influenced by that.
If you figure out how to control for publication bias
Please let me know
If you figure out how to quantify the different biases in the underlying studies I would also be interested
IMO, the best metsanalysis yields is a guess. I think it is used to often and the results are
Taken too seriously
J
Jay: you might be interested in the p-curve (http://p-curve.com/)
The biggest problem with Publication Bias is that this Bias can be multi-factorial in nature
1. The editor rejects because the paper doesn't suit the Journal
2. Scientific Bias - The reviewers have a stake in rejecting the paper
3. The Reviewers didn't fully understand the paper
4. The reviewers are inadequate
5.. etc
IMO, the use of any statistic that tries to control for Publication Bias is as valid as a Chrystal Ball
AS a teacher once told me: Statistics are only a tool and NOT an end by itslef
True Bias cannot be controlled for due to the fact that it is impossible to measure
Just my 2 cents
Jay
Thank you all for your reply!
I went through personal simulation and yes, it affects also metaregression indeed. Just in a more complex way than those reported in the paper I linked.
About controlling biases statistically, well, IMHO it is better than nothing, especially when done in a wise and smart way.
The problem is that it should be clearly understood that we never get to a proof (this is not matemetatics), but we try keep on accumalating evidence in order to embrace the probabilistically best answer we have at a certain moment. If we are fully aware of this, in the end, it's better than randomness and subjective intuition.
Massimiliano:
I disagree with your statement::
" About controlling biases statistically, well, IMHO it is better than nothing, especially when done in a wise and smart way".
Since if Bias can be measured it would be called a confounder. My question to you is how can something that is unknown be quantified.
Bias is caused by improper study design and therefore the effect of such Bias can never be known.
Statistics cannot remedy this situation.... Just can't do it..
As for your " Better than nothing" statement science does not work on a "Better than Nothing" principle
Just my 2 cents, again
Jay
- Badges
- Science topic
- Similar topics
- Medicine
- Physiology
- Biosignals
- Biomedical Signal Processing
- Statistical Analysis