Eivind, have you seen the systematic review by Kung et al (2012)? It evaluates the quality of CPGs and how it has changed over the past decade. Although not using AGREE but using the (American) Institute of Medicine standards, which is another set of quality criteria for guidelines. Interestingly, overall guideline adherence to these standards was not very high, less than 50%, and, surprisingly, the quality hadn't improved much over the last decade.

My two cents on your question is that the difference between the two versions of AGREE is (as you say) very minor so it seems to me you should be able to compare research that has used either one. In AGREE II, the scope was broadened so that it can be applied also to non-clinical guidelines, e.g. for public health, policy, etc. For example, the term patient was replaced by target population. However, I am not aware of any published comparison between the two. Hilsen! :)

It would be somewhat more accurate to say that the differences are modest but not minor, as can be seen from the brief summary comparison I provide below:

The AGREE II instrument still consists of 23 items organized into 6 domains, with many of the items now reworded from their original form in AGREE for greater specificity and clarity of usage, but there are several non-trivial differences:

(1) in domain 2 (stakeholder involvement), pilot testing of the guideline was removed;

(2) in domain 3 (rigor of development), evaluation of the strengths and limitations of the body of evidence was introduced;

(3) the original AGREE 4-point rating scale was expanded to a 7-point scale in order to enhance both reliability and discrimination;

(4) AGREE II introduces 2 global rating items: overall quality of the guideline and recommendation for its use.

As to compatibility, there have been no compatibility studies in which AGREE/AGREE II compatibility was an explicit endpoint, but nonetheless unplanned comparisons, although possibly uncovering non-material differences, have nonetheless failed to find clinically relevant differences between the guideline scores generated from AGREE versus AGREE II. So for example, Canadian researchers* compared international breast cancer guidelines across the United States (ASCO guidelines), Canada (CCO guidelines), and the United Kingdom (NICE guidelines) to evaluate the quality and consistency of content using the AGREE instrument. As a secondary analysis, the investigators compared the AGREE II instrument with the original AGREE instrument, finding that the scores obtained in their study - which were based on the original AGREE instrument - would not have significantly altered if AGREE II had been used instead, and moreover, that the differences (1) would not have been large enough to change any of the conclusions they derived under AGREE, and (2) that the variability they observed in the quality of breast cancer guidelines internationally would be maintained as per their study. Other studies that have occasionally examined this issue incidentally have been comparable, with to date no known appreciable and clinically relevant differences being detected.

* Hogeveen SE, Han D, Trudeau-Tavara S. Comparison of international breast cancer guidelines: are we globally consistent? cancer guideline AGREEment. Curr Oncol 2012; 19(3):e184-90.

Thank you both, Susanne and Constantine. I agree (!) with both of you regarding similarities between the two versions. The manual for the 2nd version describes in detail the changes. Although the number of items and domains remains the same, some items have been put into other domains and so on. Still, one might think that in a practical sense it should be acceptable to compare scores across the two versions. I will study the suggested references with interest.

Hello Eivind,

Have you seen the Australian iCAHE Guideline Quality Check List? (http://www.unisa.edu.au/Global/Health/Sansom/Documents/iCAHE/Guideline%20Clearing%20House/iCAHE%20Guideline%20Quality%20Checklist.pdf)

It has recently been preliminary validated against AGREE 2 and found to correlate quite well (http://www.ncbi.nlm.nih.gov/pubmed/24885893). It substantially shorter and quicker to use, and more clinician oriented.

Have a look and let me know what you think!

Dear Susanne. Thanks for your efforts and enthusiasm. Any tools that aim to take less resources to complete than AGREE-II will always be of interest. Regrettably this alternative tool does not seem to include any items that address implementation and measuring the use of the guideline. I think this is a problem, because to my knowledge this is one of the domains where guidelines receive the lowest scores (http://www.ncbi.nlm.nih.gov/pubmed/21127089). 

Dear Eivind,

Thanks for your comment! I agree that many guidelines lack a section on implementation and that implementation is paramount to get the recommendations of the guideline into practice. I believe, however, that the developers of the iCAHE tool feel that applicability and implementation can be better addressed outside of the actual guideline, since they strongly believe that the guideline needs to be contextualised to the local practice setting before it can be implemented. They therefore purposely left out the applicability domain from their instrument. The article I referred to discusses this briefly. Also, the instruments might serve different purposes, or target different audiences. Thanks again for your interest.