You cannot do a meta-analysis without doing a systematic review. A systematic review always precedes a properly done meta-analysis. Consult the Cochrane Handbook for Systematic Reviews of Interventions, i.e., the gold standard for procedures in doing a systematic review and meta-analysis.

Do you consider always better a meta-analysis than a systematic review? or maybe in many instances, a systematic review without the meta-analysis constraints reflects much better the reality?

If you are asking me, Jose A Calbet , I much prefer a rigorous systematic review followed by a meta-analysis compared to a systematic review alone. I am an engineer by training and thus prefer numbers. In my opinion, it is harder to lie with numbers than with words, even if the words are coming from a systematic review.

Gordon L Warren, Yes I was asking you. Thank you for your response.

In my admittedly personal opinion, the meta-analysis is the single most misused article type in sports science today. It is a cheap way for unscrupulous and dishonest researchers to make a career. Below I will attempt to outline why I detest ~80-90% of the meta-analyses that are published or at least find the majority of them to be of little use and novelty.

The modus operandi for certain "scientists" in certain subfields of sports science is evident: do one meta-analysis after the other, as many as you can in a given time, and you have soon established "authority" in the chosen field(s). You do not even have to do e.g. a training study of your own, or at least only a few now and then just to keep up the appearance that you are a real scientist.

In todays publish-or-perish climate, you can easily get away with having a ratio of say 70-30 or 80-20 between the meta-analyses and the original papers in your publication list, and chances are that your institution will think you are a very good worker and pat you on your back if you crank out meta-analyses one after the other. I personally find that even a ratio of say 50-50 is far too high.

Let us be honest: a meta-analysis, once you have learned the procedures and in addition the rules of "the publishing game", is a rather quick and brainless task that can be accomplished in a few weeks. Especially if there are several of you who write the meta-analyses and divide the sub-tasks between you (which is the norm, as nobody seems to care one iota about the Vancouver rules of co-authorship). A training study takes far longer time, even considering that the duration of training studies is nowadays typically only between 2-8 weeks. Which type of paper results in the most number of citations? Which type of paper is done in the fastest time? The answer to both questions is almost inevitably: the meta-analysis. This is why I call it a cheap way of making a careeer.

This modus operandi employed especially by certain mass-producers of papers was just about impossible 20-30 years ago, but has become not only possible but frequent during the past 10-12 years due to the explosion in the number of available journals to publish in, and due to the fact that few journal editors and reviewers bother to check whether the authors are really experts in the field. As long as various guidelines (Cochrane etc) and various bla bla bla formalities are fulfilled, and the article has a selling title, everything is assumed to be fine and dandy.

I am exaggerating a bit of course, but not very much.

Finally, and perhaps most important: the meta-analysis can never ever replace a really large randomized controlled trial. This was recognized already in the mid-1990s, when meta-analysis started to become the "gold standard" type of review. I believe that warnings such as this BMJ Editorial from 1995 are still very relevant today.

Misleading meta-analysis | The BMJ

@ quite interesting view. I have been always thinking that to review an issue the first rule is to know very well the field. Even in the most strict meta-analyses, there is always some subjectivity, the problem is how informed and biased is this subjectivity? Thus, will it not be better a systematic review or a narrative review by expert scientists, with many original articles in the field reviewed than a meta-analysis by authors without experience in the field?

Jose A Calbet and Gordon L Warren this is a very interesting discussion. I think the critical point here is "standardization" of the criteria and mechanisms to choose the articles to include (both in a systematic review and in a meta-analysis). Do you think all articles have the same weight? I think no. Although in principle any article published under peer review process should meet high credibility conditions, this does not always happen, due to the affinity of the reviewers, due to the importance of the authors' surnames, or for other multiple reasons such as an increasing difficulty for the recruitment of expert reviewers.

Open access publication should (in theory) facilitate the publication of studies with negative results and provide greater objectivity in any knowledge area, but in contrast, and this is a purely personal perception, it is simply increasing the background noise, sometimes by mechanisms such as those described very well by Mathias Wernbom.

May be I'm openning another debate, but we will probably see in the near future notable changes in the model of scientific publishing and the evaluation of scientific careers. Or we will be buried in information of doubtful value and only AI (with all the risks that this entails) will take over the personal management of bibliography and citations.

I share the opinion of Mathias Wernbom and Gines Viscor on this matter. The only relevant reference of a scientist is his/her consistency in persuding scientific questions through original research. Only then his/her works of synthesis (review or meta-analysis) becomes valid and relevant. A ratio of 80:20 or more in favor of original research (intervention studies) should become a standard for publishing any sort of synthesis papers. With open access we are getting less quality and more quantity, which slowly drowns the relevant data. A meta-analysis of false data is still a false data. Thus, an opinion, even personal, of a researcher with high milage in the field is more valid to me than systemic reviews or meta-analyses.

Jose A Calbet

Meta analysis is not something that we can always do.

In healthcare research is not always possible to perform meta analysis and then other methods of synthesis must be performed.

You do a systematic review of quantitative studies and when possible you perform meta-analysis. For this you must not have heterogeneity within your studies, population, outcomes and data, otherwise meta-analysis will be biased.

There is a very good guideline published last year to use other types of synthesis when meta-analysis is not feasible. Please see below.

https://www.bmj.com/content/368/bmj.l6890

I personally consider the terms systematic review and meta-analysis are misinterpreted in this discussion. A meta-analysis is a statistical method used to expand or to complement systematic review (if required and not always needed) so you can't have one without the other. Gordon L Warren explained it well in one short sentence.

For you to perform a meta-analysis, you have to explore something that you have identified in your systematic review. What you are exploring is heterogeneity within and between the studies included in your systematic review. When there is large heterogeneity present (and not low as stated by Helena Rezende ), you have to explore it to determine its origin by identifying moderators and then you analyze their influence on the effect size. When there is low heterogeneity or homogeneity, there is nothing to explore since your data are too similar and hence, there is no need for meta-analysis and you finish your paper at the level of a systematic review. For this, you also perform a publication bias test to explore if the studies were too selective regarding reporting of the results (this way you address reporting bias). If all of these tests show diversity between the studies, you have justification for your meta-analysis.

The paper in the link provided by @Helene Rezende, actually discusses the shortcomings of reporting methods used (such as PRISMA) in a systematic review alone and how it does not adequately address data divergency due to lack of any statistical analysis when not followed by meta-analysis.

However, like said, not every systematic review can be complemented by a meta-analysis.

There are different types of meta-analysis (like multilevel or multivariate or meta-regression) depending on the data distribution within the studies but in the end, they all have the same purpose, to explore divergence within and between the studies.

The original aim of the systematic review was (is) to reduce the quantity of research and provide quality. By performing a systematic review, we identify what is know and what is unknown and hence, have a clear justification for our research.

So, systematic reviews are an excellent tool to help us identify and future research fields and topics.

Jose A Calbet I also suggest another information to help the understanding of heterogeneity within the studies and clarify what I have mentioned in my post above and Ratko Peric have argued about it. I hope this helps. Good discussion we have here. Well done!

https://handbook-5-1.cochrane.org/chapter_9/9_5_1_what_is_heterogeneity.htm

Helena Rezende and Ratko Peric, thank you for these very interesting comments, nowadays that we are seeing many meta-analyses published. A lot of expertise is required to establish and correctly apply the inclusion/exclusion criteria for studies and to interpret the role of potential moderators and causes of heterogeneity. I should recognize that, personally, many times I feel more comfortable with systematic reviews or even narrative reviews by Great experts in the field than some meta-analyses. Besides, without mechanistic experiments is always difficult to know the real influence of potential moderators and how to account for them.

I strongly recommend this reading: “The truth wears off”, in the link:

https://www.newyorker.com/magazine/2010/12/13/the-truth-wears-off

Good discussion we have here. Well done!

I got a lot new knowledge here.

thanks for all for sharing

best regards

MUIN