Usually, the journal that asked for your review, provides a guideline of how they expect the review to be done also raising questions that the reviewer should check for in the manuscript under review.

It depends on what field you are doing the review in, whether you accept the hierarchy of evidence that sees RCTs as the gold standard. It probably also depends on what your question driving the review is. Here's one for the social sciences.

Petticrew, M., & Roberts, H. (2006). Systematic reviews in the social sciences: A practical guide. Oxford: Blackwell Publishing.

Kon's suggestion is good too and a lot more recent.

see the links below for guidance, tips, tools and resources:

http://guides.lib.unc.edu/c.php?g=148913&p=979538 for systematic review

http://guides.lib.unc.edu/c.php?g=8369&p=2634604 for literature review

https://www.cincinnatichildrens.org/service/j/anderson-center/evidence-based-care/legend

https://www.dlsweb.rmit.edu.au/lsu/content/2_assessmenttasks/assess_tuts/lit_review_LL/writing.html

The Johns Hopkins course on coursera is very good for learning how to do a systematic review.

https://www.coursera.org/learn/systematic-review

Daniela:

You have already received some fine advice. Since I am an evidence-based medical researcher (in advanced oncology), systematic reviews (SRs) (along with meta-analyses (MA) and critical appraisals) are my regular tools (including in preclinical research), the following - culled from the resource and reading list of my instructional sessions on these issues - represent a dozen key resources that can be used to achieve and sustain a high degree of facility in the conducting of SR and/or MAs, and to help avoid some of the critical limitations and pitfalls associated with both. This is not meant to be a substitute for the many excellent suggestions already rendered by the above contributors, only to supplement and hopefully enrich the guidance.

A. SYSTEMATIC REVIEW AND META-ANALYSIS PROCEDURES - THE AUTHORITIES: IN DEPTH

Detailed, practical real-time step-by-step guidelines rather than the loose outer framework of what known as the PRISMA approach. The best of these are (in my descending order of preference):

1. The Joanna Brigg's Institute Reviewer's Manual

Detailed step-by-step procedures, from The Joanna Briggs Institute (JBI) at the University of Adelaide, a

Cochrane Collaboration and the Campbell Collaboration partner.

At: http://www.joannabriggs.edu.au/Documents/JBIReviewManual_CiP11449.pdf

2. CDR's Systematic Reviews: CDR's Guidance for Undertaking Reviews in Health Care (3rd Edition)

This excellent detailed handbook of the methods and steps necessary to conduct a systematic review

is from the authoritative Centre for Reviews and Dissemination (CRD) at the University of York who publish the DARE (Database of Abstracts of Reviews of Effects) Database of SRs.

At: http://www.york.ac.uk/inst/crd/SysRev/!SSL!/WebHelp/SysRev3.htm

3. Cochrane Handbook for Systematic Reviews of Interventions

From the renown Cochrane Collaboration, this is the most detailed but you may find the other two somewhat easier to commence with, then graduate to this handbook for all the advanced refinements, should that prove necessary, as a reference work although it has exceptional riches.

At: http://handbook.cochrane.org/

B. SUPPLEMENTARY RESOURCES:

4. Systematic Review and Meta-analysis Methodology

This article represents an excellent detailed exposition of all the critical procedures of conducting SRs and MAs. Uncommonly wise on the key issue of study combinations, with invaluable tips and guides on the Mantel-Haenszel method and the Inverse Variance method for data combination, and constructing a Forest Plot, on measuring heterogeneity, performing sensitivity analyses, on assessing publication bias through Funnel Plots, and on the use of the automatic SR/MA software tool RevMan, discussed below. All told, an extraordinary contribution.

[Crowther M, Lim W, Crowther MA. Systematic review and meta-analysis methodology. Blood. 2010 Oct 28;116(17):3140-6.]

At: http://bloodjournal.hematologylibrary.org/content/116/17/3140.long

5. Assessing Heterogeneity Of Primary Studies In Systematic Reviews And Whether To Combine Their Results

Fine tutorial on dealing with the issues of pooling and heterogeneity, with valuable tips and examples, from the on-going series of tutorials from the Evidence-Based Medicine Teaching Tips Working Group in Canada.

[Hatala R, Keitz S, Wyer P, Guyatt G, Evidence-Based Medicine Teaching Tips Working Group. Tips for learners of evidence-based medicine: 4. Assessing heterogeneity of primary studies in systematic reviews and whether to combine their results. CMAJ 2005 Mar 1; 172(5):661-5]

At: http://www.cmaj.ca/content/172/5/661.full

C. CRITICAL PERSPECTIVES: HOW NOT TO DO A SR/MA, AND WHAT TO AVOID

6. The Need For Caution In Interpreting High Quality Systematic Reviews

This key paper on the highly variable quality of systematic reviews themselves, makes clear and addresses the dilemma, with an illustrative case study, that two or more systematic reviews on precisely the same question not infrequently arrive at opposite conclusions, raising questions both of the validity, or the relevance, of the conclusions. Lessons worth learning.

[Hopayian K. The need for caution in interpreting high quality systematic reviews. BMJ 2001 Sep 22; 323(7314):681-4]

At: http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1121240/?tool=pubmed

7. A Guide To Interpreting Discordant Systematic Reviews

The classic paper on how to resolve discordance across systematic reviews from the eminent expert Alejandro Jadad (at McMaster), a must read.

[Jadad AR, Cook DJ, Browman GP. A guide to interpreting discordant systematic reviews. [Journal Article]

CMAJ 1997 May 15; 156(10):1411-6]

At: http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1227410/pdf/cmaj_156_10_1411.pdf

8. Trials and Tribulations of Systematic Reviews and Meta-Analyses

Good treatment of some of the major limitations besetting systematic reviews (including limited datasets, unpublished data, and both statistical and clinical heterogeneity).

[Crowther MA, Cook DJ. Trials and tribulations of systematic reviews and meta-analyses. Hematology Am Soc Hematol Educ Program 2007; :493-7]

At: http://asheducationbook.hematologylibrary.org/content/2007/1/493.long

9. Appraising Systematic Reviews and Meta-analyses

Provides criteria to use to critically appraise systematic reviews and meta-analyses, and documents some of the key pitfalls of both.

[Bigby M, Williams H. Appraising systematic reviews and meta-analyses. Arch Dermatol 2003; 139(6):795-8]

At: http://archderm.jamanetwork.com/article.aspx?articleid=479334

10. Reading And Critically Appraising Systematic Reviews And Meta-Analyses

The cautions needed in reading and conducting a systematic review, especially strong on sensitivity analysis, meta-regression, sub-group analysis, publication bias and missing data.

[Balk EM, Lau J, Bonis PA. Reading and critically appraising systematic reviews and meta-analyses: a short primer with a focus on hepatology. J Hepatol 2005; 43(4):729-36]

At: http://www.journal-of-hepatology.eu/article/S0168-8278%2805%2900489-7/fulltext

D. TOOLS AND UTILITES

11. RevMan

RevMan, a free review tool the Cochrane Collaboration Group, is used to facilitate literature reviews and the meta-analyses conducted within the Cochrane Review Protocols. Download latest version from:

http://ims.cochrane.org/revman/download

One of its key virtues is that it can be easily used by the medical researchers who aren't statisticians, although for those who are, it's an accessible tool for performing the meta-analyses and for generating the graphs (forest plot, funnel plot) in publication standard. It uses both fixed and random statistical models, the latter being the leading ones, namely the DerSimonian and Laird random-effects models.

Although relatively straightforward, I would still advise working through the tutorials, tips, and webinars that the Cochrane Collaboration Group provide as part of their documentation and their Open Learning Material. All these are accessible at:

http://ims.cochrane.org/revman

In addition, I would strongly suggest the excellent introduction to RevMan for New Authors produced as a video by John MacDonald of Cochrane Canada, available here:

http://www.youtube.com/watch?v=Mu5P3_3OJ8U

Finally, remember that to conduct a complete meta-analysis, there's much to be done prior to entering the info and data into the RevMan, especially organizing the proper literature search. It takes not a trivial amount of time to learn and master RevMan but it can be a valuable aid in producing high-quality SRs and MAs.

12. Systematic Review Data Repository (SRDR)

Online tool for the extraction and management of data for systematic review or meta-analysis, which also serves as a Web-based repository of systematic review data, provided as a joint initiative of the Agency for Healthcare Research and Quality (AHRQ) and the Evidence-based Practice Center (EPC) at Tufts Medical Center.

At: http://srdr.ahrq.gov/

FINAL CAUTION: THE ABSENCE OF CRITICAL APPRAISAL

The methodological assessment that go into most SRs and MAs can systematically fail to capture intrinsic flaws in the underlying RCTs meeting the inclusion criteria.

Consider one example: the study that is most frequently cited to support the detrimental effect of antioxidant supplementation concurrent with oncotherapy is the Laval University RCT in head and neck (HNC) patients [Bairati I, Meyer F, Jobin E, et al. Antioxidant vitamins supplementation and mortality: a randomized trial in head and neck cancer patients. Int J Cancer 2006;119:2221–4], and several SRs on the question of adverse effects of antioxidant therapies on oncologic survival favorable include this RCT, rating it as high quality. But when subjected to intensive critical appraisal, this trial is found to be profoundly - and indeed fatally - methodologically compromised and cannot support the conclusion of harm that it drew as to adverse interference of antioxidants on the efficacy of radiation therapy. The reason for this is that upon appraisal we discover that the only increase in risk for mortality in the RCT was limited to patients who smoked during radiation therapy, where we must note the extraordinary oddity and laxity of protocol design where subjects were allowed to smoke in a trial evaluating the interaction of antioxidant supplementation concurrent with radiotherapy, when clearly smoking is a powerful confounder, and smoking was in fact the driver of the increased mortality, not antioxidant supplementation [Meyer F, Bairati I, Fortin A, et al. Interaction between antioxidant vitamin supplementation and cigarette smoking during radiation therapy in relation to long-term effects on recurrence and mortality: a randomized trial among head and neck cancer patients. Int J Cancer 2008;122:1679–83]. None of the SRs on this issue stumbled to that fact, not discoverable as part of SR/MA methodological assessment, which is different profoundly from intensive critical appraisal of individual studies.

Consider a second example: numerous SRs and MAs have concluded that the evidence unambiguously supports the contention of an obesity paradox wherein being overweight is claimed to be associated with lower mortality compared to normal weight subjects (I am simplifying somewhat for the sake of discussion, but in no way that materially affects my points). However what all these pro-obesity paradox SR/MAs missed was that all the RCTs assessed used BMI as the weight metric, yet critical appraisal of this issue indisputably establishes (as I conclude in my review of the obesity paradox, available on my ResearchGate Profile) that BMI is a wholly and demonstrably inadequate metric of adiposity, and that upon such appraisal and re-review of all the studies on the question of the existence of an obesity paradox, support for a clinically relevant obesity paradox evaporates.

I could conjure dozens of other examples (even involving Cochrane Reviews) where the failure to conduct intensive individual and cross-literature has lead to false and misleading conclusions in SRs and MAs undertaken. What all this however collectively demonstrates, along with the other limitations discussed in the literature above, is that systematic reviews and meta-analyses are not auto-validating technologies and that many truths can and are missed by SR/MAs and many misleading conclusions illicitly derived. There is simply no substitute for critical thinking and critical appraisal outside the borders of SR/MA protocols to avoid the disservice of derived false conclusions, which advances our scientific knowledge not one whit, which after all is the point of conducting systematic reviews and meta-analyses.

Constantine Kaniklidis

Director of Medical Research, No Surrender Breast Cancer Foundation (NSBCF)

Oncology Reviewer, Current Oncology journal

Thank you very much! You helped me a lot with these answers. 

All the best!

Daniela

Constantine, you have given an overwhelming amount of references which will prove very useful to many. Much appreciated