I have never read a systematic review or meta-analysis where a study was excluded due to not being pre-registered as is required by the Declaration of Helsinki. Is this a matter of "the ends do not justify the means" or is it not one to be concerned about?
I think it should not be included because it has not met the ethical requirements, so how will the information collected from the participants be protected and who will take ownership of that information.
Pre-registering is more often required for a systematic review/meta-analysis and not a primary research study that could be in it. Yes, it is becoming more common to pre-register a study/trial now a days but if you exclude studies that are not pre-registered, you may simply just be excluding the older studies. They would be punished simply because pre-registering wasn't around when they were conducted. Thus, requiring pre-registration could impose a bias in your meta-analysis. An approach to evaluate the effect of only including pre-registered studies vs. all studies in a systematic review/meta-analysis would be to run a subgroup analysis comparing pre-registered vs. unregistered studies.
I would also consider using meta-regression including both pre-registration status and time elapsed since publication.
After performing a meta-analysis, it is advisable to study the influence of each of the studies on the results obtained. Sensitivity analysis involves replicating the meta-analysis results by excluding one of the studies included in the review at each step. If the results thus obtained are similar, both in direction and magnitude of the effect and statistical significance indicates that the analysis is robust. This same process could be repeated by eliminating at the same time several studies (e.g. those of poorest methodological quality, unpublished ones, etc.) to determine their possible influence on the results.
Along with the sensitivity analysis, once the results of the meta-analysis have been obtained, the existence of a possible selection bias should be analyzed that could call into question the results achieved. Among the most commonly used methods to assess the existence of such biases the most popular is the funnel plot, which is based on representing the sample size of each job versus the size of the detected effect. It would be normal for all studies to detect an effect of similar magnitude, around a horizontal line, with greater dispersion the smaller the sample size. In this way, the points would tend to be distributed in the form of an inverted funnel. If, on the other hand, there was a publication bias, only those that found significant differences between the groups would be published from the smaller sample-sized studies, so that the point cloud would appear deformed at one end. There are other statistical techniques such as the Begg or Egger test, implemented in most meta-analysis programs, which allow to evaluate in a more objective way the existence of a possible publication bias.
- Badges
- Science topic
- Similar topics
- Philosophy
- Ethics