Hi,

This scale is now frequently used in both observational and cross-sectional studies. Many papers report them. The meta-analysis also uses them.

As a reviewer, I often find that the NOS leads to researchers overestimating the quality of the included studies, in other words underestimating risk of bias, which is quite worrisome.

I recommend the AHRQ Methodology checklist for prevalence studies http://www.ncbi.nlm.nih.gov/books/NBK35156/

Or the JBI tool for critically appraising studies that report prevalence data

https://wiki.joannabriggs.org/display/MANUAL/Appendix+5.1%3A+Critical+Appraisal+Instrument+for+Studies+Reporting+Prevalence+Data

Yes I have used the NOS in a good number of meta-analyses. I think Victor brought up an important point though -- the scale does tend to be a bit lenient. I have used both ROBINS-I and NOS, and find the ROBINS-I to be more detailed and less lenient overall.

Anyhow, Cochrane recommends ROBINS-I for non-randomized studies of interventions, and mentioned NOS as an alternative (https://methods.cochrane.org/methods-cochrane/robins-i-tool), so I think both are fine. I have not had any journal / reviewer criticizing the use of NOS anyway.

Hope this helps.

A usual practice while applying NOS is not providing exact criteria for judgments in a particular NOS domain. For example, confounding - one researcher might judge age- and sex-adjustment as sufficient to score this domain for 1 (or star in original adaptation); another researcher can create a tailored list of essential confounders and compare each study against this list (similarly to the ROBINS tool).

ROBINS-I (there is also ROBINS-E for exposures; however, both are used interchangeably in SRs covering observational studies) is a more detailed tool, which is harder for researchers new to SRs. Therefore, NOS is still popular; however, its use should be reported in detail, allowing transparency and replicability of quality assessment.

Thank you very much! Jeffrey Shi Kai Chan Victor Volovici Jakub Morze and Bhogaraju Anand