When conducting a meta-analyses, all the works included are rated on the basis of their methodological qualities. Which rates, scales, or criteria should be used? Does anyone have a rating scale available?
Thanks for your suggestion, Julius. I would like to analyze the effects of intervention, so some of the criteria are a) case-control studies, b) n>= 10 in all the groups, c) outcome variables regarding memory (not MMSE, MoCA,...).
I agree that the STROBE is a good choice in your case. I would suggest not to use the quality rating for inclusion/exclusion purposes. Rather, I think it is better to include this information in a moderator analysis (if you have enough studies to do it)
Thanks for your suggestions. What I am doing is selecting articles based on some criteria (outcomes, case-control, sample sizes, etc). Among those included, I check individually each of the points, for them to be compared a posteriori ans give details about the quality of studies.
For reviews of RCTs: The Cochrane Collaboration has an excellent handbook available online (http://www.cochrane.org/training/cochrane-handbook) which I can recommend. Cochrane has their own rating scale for RCTs looking at internal and external validity, see their handbook. They also provide RevMan5, free software for preparing and doing a review (http://tech.cochrane.org/revman). Another rating scale is the PEDro scale, developed by the Physiotherapy Evidence Database (http://www.pedro.org.au/english/downloads/pedro-scale/).
Dear Morten, thank you a lot for your recommendations. I have reading about Cochrane collaboration handbook and RevMan, but you can not use it unless you are registered and have an account in Cochrane. I read about PEDro scale long ago, and totally forgot. I will take a look at it again. I am collecting all these scales to find which one fits the best in my research. Thank you very much for your help!
I agree that either the Cochrane Risk of Bias or the Strobe cheklist are good choices. However, if you want another option you can consider the Down´s and Black checklist:
is a strong and valid tool. One of the disavantages is that it´s very long but is a much flexible checkclist (i mean that can be used in different study designs).
Finally, the PEDro Scale could be usefull also, it´s "smaller" (11 items), is valid and avaliable in different languages:
Keep in mind that the STROBE checklist is a reporting checklist and what you want in meta-analysis is a risk-of-bias (methodological quality) checklist so STROBE is not suitable. There is a paper that may be helpful: Sanderson S, Tatt ID, Higgins JP. Tools for assessing quality and
susceptibility to bias in observational studies in epidemiology: a systematic
review and annotated bibliography. Int J Epidemiol. 2007 Jun;36(3):666-76. Epub
Try AMSTAR. See my paper entitled, "Meta review of systematic and meta analytic reviews on movement differences, effect of movement based interventions, and the underlying neural mechanisms in autism spectrum disorder." for reference.
The choice of your assessment tool will strongly depend on the studies included in your review. Have a look at the EQUATOR network (http://www.equator-network.org/), since they provide a summary of most available tools to assess methodological quality of included studies.