I don't know of  a specific scale, but researchers often develop criteria for selection and inclusion of studies. There is a paper by Maaike van der Noordt et al. that looks at the health effects of employment which includes a systematic review of prospective studies published in Occupational and Environmental Medicine in Feb 2014 that may be helpful. You might also look at Cochrane systematic reviews although the focus tends to be on interventions. There are some tools for diagnostic accuracy such as the Quality Assessment of Diagnostic Accuracy Studies (QUADAS). A paper by Garg et al. Systematic Review and Meta-analysis: When One Study is Just not Enough published in Clin J Am Soc Nephrol 2008 talks about the issues and in the references there are some interesting examples.  Others may know if there are scales out there. I hope this helps.

Lots of people use the Newcastle Ottawa Score, but there are quite a few to pick from. Look up Sanderson S, Tatt ID, Higgins JPT. Tools for assessing quality and susceptibility to bias in observational studies in epidemiology: a systematic review and annotated bibliography. International Journal of Epidemiology 2007;36:666–676. It's a bit out of date, but a good starting point for you. But the general advice is not to think of these as scales producing one overall rating, but areas of potential risk of bias, and report on each area.

I agree with Toni's response above.  Memorializing pre-defined inclusion and exclusion criteria are important.  Depending on the type of retrospective study you are looking at there are other options for more objective grading criteria.  QUADAS is good for diagnostic studies, but if you are looking at epidemiologic studies, that wont work.  You may want to look at Strengthening the reporting of Observational studies in Epidemiology (STROBE) criteria or Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA).  While PRISMA does focus on RCTs, not epidemiological studies, it does have some information on what should be included in a systematic review.

http://www.strobe-statement.org/

http://www.prisma-statement.org/

STROBE and PRISMA are reporting guidelines. MOOSE is the equivalent for meta-analysis observational studies, which yours presumably is. So, for example, PRISMA recommends authors assess risk of bias, but does not assess risk of bias itself. For "risk of bias" read "quality of research". So both aspects are important, but quality of reporting is different from quality of the research. Which were you interested in ?

Thanks Mr. Darren, my interest is for systematic reviews.