Minimum Number of Studies for Systematic Reviews

You may write/publish a protocol for a systematic review and then find no studies! If you follow the systematic review process and after systematic search you find no studies, it is still a systematic review and worth publishing. It is still an evidence showing there is no rigor evidence or at least if there is, you couldn't find it following a systematic process. So as Priyanka Kapoor mentions below, what makes it a systematic review is a systematic process not the number of included studies.

There are published protocols of systematic reviews in Cochrane Library with no included studies. In one of the systematic reviews I remember the authors found no studies and then they actually did the first RCT in the topic and then included it in the systematic review!

There is no limitation in terms of number of included studies, however, while publishing your review in the journals, they might apply subjective criteria and publish the systematic reviews with more than one included studies. In addition, there are systematic reviews published in journals targeting an important question but they found no evidence; some times they call it 'failed systematic review', I disagree. I also disagree calling them Zombie Reviews or Empty Reviews because these names are not futuristic.

Depending on the protocol of your systematic review, you can include any study design as far as you consider limitations in each design and not to mix apples and oranges in assessment and analysis.

Minimum Number of Studies for Meta-Analysis

Bear in mind, if you want to do a meta-analysis then if the quality of studies are good enough, enough number of people or events, and as Patompong Ungprasert mentions below having 5 studies might be enough for a meta-analysis; however, to assess publication bias you will need at least 10 studies.

I agree with Farhad- there is no minimum number and the results of the review will reflect the paramters of your process. If you exclude certain research types you will limit the available number.

If you do include a number of different research types you do need to reflect the fact that some research types are higher in the heirarchy of evidence. I attempted to weight evidence on heirarchy and also reporting quality in my Master's thesis. Not sure how successful that was- I have attached it so you can see for yourself. I have also attached a link to an article demonstrating how to write a lit review with minimal papers.

Good luck!

David

Article Special observation and older persons with dementia/delirium...

Thesis Manual hyperinflation as a therapeutic technique in the prev...

Thank you David and Farhad! My belief was that a systematic review is a ( review ) of current papers, so it has to summerize many papers in certain topic. That is different for systematic review of thesis ( master or PHD) because you can introduce new ttt or diagnosis so there is no literature about the ttt or the technique

There is no magic number but I think that you cannot make any meaningful conclusions with fewer than 5 studies.

I agree with Farhad and with David: there is no "a priori" number of papers, but be aware that a systematic review is not a review of current papers. It is systematic because you have to look for ALL the published papers (current or not, anyway reporting inclusion and exclusion criteria, among them maybe year of publication).

If you are interested in treatment you have to do only with longitudinal studies, prospective or retrospective, because other designs, namely case-control (even with statistical techniques like propensity score) carry relevant risks of biased results.

Be aware of ambiguous labels like "quasi-experimental design" or "pragmatic": you have to read the methods to discover what is meant. 

Other reviews cannot bring original results and can contribute only with their references to scroll with manual search for papers not in a PUBMED query result.

Case report can be useful for stimulating research but are at the very bottom of an evidence-based conclusion because their scientific meaning is very low (except for a n-1 trial, in other words in the case of trials with only one patient and a sort of reated cross-over trial -a rather neglected design).

Completely agree with the above. My question, should we rely on the validated filters (for searching RCTs or other studies) from SIGN or BMJ in the systematic search process?

Dear Anirban,

If you are following Cochrane style, I suggest you use Cochrane search filters for RCT too: http://handbook.cochrane.org/chapter_6/6_4_11_search_filters.htm

RCT Filters for both MEDLINE and EMBASE have been tested, you may check the references at the end of Search chapter.

May be you will find useful to check the Prisma Grid, that you can find as other useful research guidelines for example on Equator Network

http://www.equator-network.org/reporting-guidelines/prisma/

Thanks fellows for every knowledgeable answer to a keen researcher who could not find proper correct guidance here. In fact what they find is someone telling them " if no studies are found then it is not a systematic review", "if you find studies but you could not get data to perform a meta-analysis then it is not a systematic review", ... etc

I really appreciate all the comments and feedbacks given by stalwarts here. You have exactly answered what I needed to know. I have already done two systematic reviews, one published and one unpublished after registration in PROSPERO. Have followed PRISMA guidelines for the same.  For my knowledge I just wanted to know if , for some topic we have less than five studies available, could we still register it. But you have answered it perfectly , that it is the process that makes a review systematic and not the number of studies. thank you so much.

Thank you so much friends for all your useful comments. I just wonder, if after doing a systematic method I find only one article in my topic can I do a systematic literature based on it?

There is no minimum or maximum limit for studies to be quoted

While I have understood from this discussion that there is no like minimum number of articles (except for those who say, a review of less than 5 papers may be questionable),I was wondering if there is any authority for this. Lets say after applying all inclusion and exclusion standards and quality assessment criteria I remain with 9 articles for synthesis, how would I justify that this would make any significant contribution? Any papers published in reputable journals with papers for synthesis ranging from 5 to 50? Please help

@Joseph - it would partly depend on your topic area and discipline. But just to emphasise the general points made in the discussion, even the most rigorous reviews, like Cochrane reviews, often result in "empty reviews" with zero included articles. See the article below.

• Article Empty Reviews: A Description and Consideration of Cochrane S...

Steven, thank you so much. That was pretty much helpful

cheers Steven

Thank you all for the informative responses. I have few concerns and would like to find an answer for such: I just started the process of systematic review and I am kind of confused wether to use MeSH search of keywords or rely on General PubMed search and would that make a big difference?

Second I heard a lot about systematic review Protocol, is there any templates for such protocol so that I can at least understand it's components?

Thank you again!

Systematic review constitutes an outstanding and consistent way to survey scientific literature in order to make a wide impression on how a research question has been addressed so far. When you are interested in approaching a very new topic or intervention, you may find no hits after well-established searches, as a sign of more efforts should be made to cover concrete scientific gaps. To sum up, I think the number of studies is not as important as it could be expected but the robustness and systematicity of review protocols do.

The minimum number of articles to be included in a systematic review does matter but this might depend on the study design from which the articles are being selected from

It does not matter.The number of studies found following a systematic literature search constitutes your systematic review.

Thanks for the discussion, key thing is to ensure a thorough process

No minimumm number in a systematic review. howeve, if the review involves a meta-analysis that has groups you might want to consider the distribution of studies included.

Heba Huessin thank you for your question. May I ask why there are few studies in your systematic review? I currently have the same problem. I found only 5 studies that fit my criteria because the topic/area is recent and new. Usually we use a systematic review when we have a lot of literature and methodologies and need to synthesize it. It's not my case. Did you published your review?

Best regards,

Daniela

Hi Hiba,

The number of studies for systemic review is not important if you search all related search engines site and you could not find more studies especially if the sample size of the population is acceptable. Y

For the second part of your question of whether all studies be of same study design, this depends on your inclusion and exclusion criteria you had designed before you start your systemic review

the information and views shared are very interesting and knowledge gaining. But what if the author described the one study included in the systematic review process under results apart from the process undertaken for review. is it meaningful to publish the paper

It is meaningful to publish as it may pave way for the other researchers when they are interested to take up research in those areas. It gives them a basic idea to further proceed.....

But it can't be taken up for a meta analysis.

There is no limit in terms of no of studies, however the manuscript should follow a systematic process and follow certain set guidelines of PICO, PRISMA and Cochrane based registry.

Moreover the base of any systemic review is a simple, precise and clear cut research question to follow.

@Farhad Shokraneh how I can find the reference that said: "no minimum number of the study included for systematic review"?

Thank you for all of your valuable comments. specially to Farhad Shokraneh

1. For more clarifications -- If I have one included article from the systematic review still I can publish a paper right?

2. Question -- Can't I include published thesis/dissertations in the systematic review paper? Should it only be peer reviewed articles?

Thanks...

For your Q2 Duleeka Kuruwita Arachchige : One of the advantage in the systematic review and meta-analysis is to include all available text and paper including thesis. So, if you do this, you will increase your quality of article.

Duleeka Kuruwita Arachchige ... you need to ask yourself what value is added by your systematic review of 1 paper? Why would someone publish it? I would not go as far as to say there is no added value, but you have to be clear what that value is, is or else others may just as well read the one paper you include... in answer to your second question, you certainly can and should include so called gray literature if you can. Research should be judged on its own merit and if there is limited published material the added value of these other sources could be high.

During a session, my professor told me about one systematic review with no studies. The researcher of that study concluded by saying that there is a need for more research on similar questions.

There is no minimum number that needs to be included in a systematic review. But the higher number of studies you have, better chances of conducting meaningful analysis. Ideally including only randomized clinical trials would give you a high level of evidence. When non-randomized trials are included, the assessment of bias of the studies should be performed accordingly. I am attaching a recent systematic review and meta-analysis which may help you in the risk of bias assessment.

Article Does Mini Screw Assisted Rapid Palatal Expansion (MARPE) hav...

Systematic review is not just about the results, but its also about the process and all the steps taken. After screening items in a systematic review, and you end up with zero items for inclusion in your study, that is still a systematic review. It may mean that there are no studies on the topic/subject you are working on, that is still a finding.

In fact, there is no minimum number of studies to be included in a systematic review. Generally, the more number of studies you include in the systematic review, the more accurate your results will be.

There is no minimum number of studies to be included in systematic review, but if one wish to do meta-analysis, at least two studies are required to sum-up the data and synthesis the evidences that will add some new evidence and knowledge in current knowledge.

What a great discussion. In my experiences conducting systematic review, i face not problems not only pertain to the number of studies, but also criteria for inclusion and model/method of analysis.

If anyone can give suggestions, it would be very helpful.