Use the MASTER scale

Article The MethodologicAl STandards for Epidemiological Research (M...

Given that this appears to be a medical study I wonder if you should mix RCTs with nonrandom studies. Wouldn't two separate analyses and then a comparison of the two results be better? Best wishes David Booth

The assessment of quality essentially asks the question "what is the degree to which safeguards against bias were implemented in a study?" One category of such safeguards are design related such as randomization or blinding and therefore a unified scheme can downgrade assessments by design such that a RCT will have design element safeguards counted and the latter that do not apply to a cohort study will not be counted.

Regarding whether we can combine RCTs with say cohort studies in a meta-analysis, they both include confounding by chance and only confounding by indication is absent in a RCT as well as selection biases. Therefore a well conducted cohort study may be better than a poorly conducted RCT in terms of bias and a unified scale helps make this assessment

See this paper:

Shrier I, Boivin JF, Steele RJ, Platt RW, Furlan A, Kakuma R, Brophy J, Rossignol M. Should meta-analyses of interventions include observational studies in addition to randomized controlled trials? A critical examination of underlying principles. Am J Epidemiol. 2007 Nov 15;166(10):1203-9.

When conducting a systematic review and meta-analysis, it's important to assess the quality of the studies included to ensure that the results are valid and reliable. However, the quality assessment tool you use may differ depending on the type of study design.

For randomized controlled trials (RCTs), the Cochrane Risk of Bias tool is widely used to assess the quality of the studies. It assesses the risk of bias across several domains, including random sequence generation, allocation concealment, blinding of participants and personnel, blinding of outcome assessment, incomplete outcome data, selective reporting, and other biases.

For non-randomized studies such as cohort studies, the Newcastle-Ottawa Scale (NOS) is a commonly used tool to assess quality. It assesses the quality of the study based on three main domains: selection of study groups, comparability of groups, and assessment of outcomes.

While the NOS is a qualitative tool, it can be converted to a quantitative tool by assigning scores to each item. However, it's important to note that using a single tool to assess the quality of both RCTs and non-randomized studies may not be ideal as the domains assessed are different.

If you still prefer to use a single tool, you can consider using the Cochrane Risk of Bias tool and adapting it to include items relevant to non-randomized studies. Alternatively, you can use the NOS for all the studies, but you would need to acknowledge the limitations of using a tool designed for non-randomized studies to assess the quality of RCTs.