Welcome to the GRADE working group

The Grading of Recommendations Assessment, Development and Evaluation (short GRADE) working group began in the year 2000 as an informal collaboration of people with an interest in addressing the shortcomings of grading systems in health care. The working group has developed a common, sensible and transparent approach to grading quality (or certainty) of evidence and strength of recommendations. Many international organizations have provided input into the development of the GRADE approach which is now considered the standard in guideline development.

Judgments about evidence and recommendations in healthcare are complex. For example, those making recommendations about whether or not to recommend a new generation of blood thinners for patients with irregular heart beat (atrial fibrillation) must agree on which outcomes to consider, which evidence to include for each outcome, how to assess the quality of that evidence, and how to determine if blood thinners do more good than harm. Because resources are always limited and money that is allocated to treating atrial fibrillation cannot be spent on other worthwhile interventions, they may also need to decide whether any incremental health benefits are worth the additional costs.

Systematic reviews of the effects of healthcare provide essential, but not sufficient information for making well informed decisions. Reviewers and people who use reviews draw conclusions about the quality of the evidence, either implicitly or explicitly. Such judgments guide subsequent decisions. For example, clinical actions are likely to differ depending on whether one concludes that the evidence that blood thinners reduces the risk of stroke in patients with atrial fibrillation is convincing (high quality) or that it is unconvincing (low quality).

Similarly, practice guidelines and people who use them draw conclusions about the strength of recommendations, either implicitly or explicitly. Using the same example, a guideline that recommends that patients with atrial fibrillation should be treated may suggest that all patients definitely should be treated or that patients should probably be treated, implying that treatment may not be warranted in all patients. A systematic and explicit approach to making judgments such as these can help to prevent errors, facilitate critical appraisal of these judgments, and can help to improve communication of this information.

One of the aims of the GRADE Working Group was to reduce unnecessary confusion arising from multiple systems for grading evidence and recommendations. To avoid adding to this confusion by having multiple variations of the GRADE system we suggest that the criteria below should be met when stating that the GRADE approach was used to assess evidence or develop recommendations. Also, while users may believe there may be good reasons for modifying the GRADE system, we discourage the use of "modified GRADE approaches" that differ from the approach described by the GRADE Working Group.

On the other hand, we encourage and welcome constructive criticism of the GRADE approach, suggestions for improvements, and involvement in the GRADE Working Group. As most scientific approaches to advancing healthcare, the GRADE approach will continue to evolve in response to new evidence and to meet the needs of systematic review authors, guideline developers and other users.

Suggested minimal requirements for stating that the GRADE system or approach was used (updated 2023-05)

  1. The certainty in the evidence (also known as quality of evidence or confidence in the estimates) should be consistent with the operationalizations used by the GWG.[1, 9-11] Conceptually, GRADE has provided the following definition “The certainty that the true effect, accuracy measure, or association lies on one side of a particular threshold, or in a particular range.”[12, 13] And for evidence from reviews of qualitative research: “an assessment of the extent to which the review finding is a reasonable representation of the phenomenon of interest.”
  2. Explicit consideration should be given to each of the GRADE domains for assessing the certainty in the evidence (i.e., risk of bias, imprecision, inconsistency, indirectness, publication bias, large effects, dose response gradients, and residual plausible opposing bias), although different terminology may be used to describe these domains (e.g., limitations in the detailed study design or execution instead of risk of bias).
  3. The certainty in the body of evidence should be assessed for each critical or important outcome for a decision (also called main outcomes) using GRADE’s categories (e.g., high, moderate, low and/or very low), with definitions for each category that are consistent with the definitions published by the GWG.[9] These outcomes should describe both desirable and undesirable health effects (also called benefits and harms or efficacy and safety outcomes).
  4. GRADE evidence tables (e.g., evidence profiles or summary of findings tables) should be used to present the results of evidence synthesis and an assessment of certainty in evidence, and these should be based on systematic reviews (we refer to systematic reviews as a methodology that is also used in health technology assessments and other evidence synthesis following such methodology).[14] At a minimum, the evidence that was assessed and the methods that were used to identify and appraise that evidence using GRADE should be clearly described.

The next three requirements apply only to GRADE for making decisions or recommendations:

How to cite this GGG guidance: Schunemann HJ, Brennan S, Akl EA, Hultcrantz M, Alonso Coello P, Xia J, et al. The development methods of official GRADE articles and requirements for claiming the use of GRADE - a statement by the GRADE Guidance Group. J Clin Epidemiol. 2023 May 19;159:79-84. doi: 10.1016/j.jclinepi.2023.05.010.

Citations:

  1. Atkins D, Best D, Briss PA, Eccles M, Falck-Ytter Y, Flottorp S, et al. Grading quality of evidence and strength of recommendations. BMJ. 2004;328:1490.
  2. Schunemann HJ, Best D, Vist G, Oxman AD, Group GW. Letters, numbers, symbols and words: how to communicate grades of evidence and recommendations. CMAJ : Canadian Medical Association journal - journal de l'Association medicale canadienne. 2003;169:677-80.
  3. Guyatt GH, Oxman AD, Vist GE, Kunz R, Falck-Ytter Y, Alonso-Coello P, et al. GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. BMJ. 2008;336:924-6.
  4. Guyatt G, Oxman AD, Akl EA, Kunz R, Vist G, Brozek J, et al. GRADE guidelines: 1. Introduction-GRADE evidence profiles and summary of findings tables. Journal of clinical epidemiology. 2011;64:383-94.
  5. Alonso-Coello P, Schunemann HJ, Moberg J, Brignardello-Petersen R, Akl EA, Davoli M, et al. GRADE Evidence to Decision (EtD) frameworks: a systematic and transparent approach to making well informed healthcare choices. 1: Introduction. BMJ. 2016;353:i2016.
  6. Zhang Y, Akl EA, Schunemann HJ. Using systematic reviews in guideline development: the GRADE approach. Res Synth Methods. 2018
  7. Dixon C, Dixon PE, Sultan S, Mustafa R, Morgan RL, Murad MH, et al. Guideline developers in the United States were inconsistent in applying criteria for appropriate Grading of Recommendations, Assessment, Development and Evaluation use. Journal of clinical epidemiology. 2020;124:193-9.
  8. McGowan J, Akl EA, Coello PA, Brennan S, Dahm P, Davoli M, et al. Update on the JCE GRADE series and other GRADE article types. Journal of clinical epidemiology. 2021;140:163-4.
  9. Hultcrantz M, Rind D, Akl EA, Treweek S, Mustafa RA, Iorio A, et al. The GRADE Working Group clarifies the construct of certainty of evidence. J Clin Epidemiol. 2017;87:4-13.
  10. Guyatt GH, Oxman AD, Kunz R, Vist GE, Falck-Ytter Y, Schunemann HJ. What is "quality of evidence" and why is it important to clinicians? BMJ. 2008;336:995-8.
  11. Guyatt G, Oxman AD, Sultan S, Brozek J, Glasziou P, Alonso-Coello P, et al. GRADE guidelines 11-making an overall rating of confidence in effect estimates for a single outcome and for all outcomes. Journal of clinical epidemiology. 2012.
  12. Schünemann HJ, Neumann I, Hultcrantz M, Brignardello-Petersen R, Zeng L, Murad MH, et al. GRADE Guidance article 35: Update on rating imprecision for assessing contextualized certainty of evidence and making decisions. Journal of clinical epidemiology. 2022.
  13. Schunemann HJ. Interpreting GRADE's levels of certainty or quality of the evidence: GRADE for statisticians, considering review information size or less emphasis on imprecision? Journal of clinical epidemiology. 2016;75:6-15.
  14. Schunemann HJ, Reinap M, Piggott T, Laidmae E, Kohler K, Pold M, et al. The ecosystem of health decision making: from fragmentation to synergy. Lancet Public Health. 2022;7:e378-e90.
  15. Alonso-Coello P, Oxman AD, Moberg J, Brignardello-Petersen R, Akl EA, Davoli M, et al. GRADE Evidence to Decision (EtD) frameworks: a systematic and transparent approach to making well informed healthcare choices. 2: Clinical practice guidelines. BMJ. 2016;353:i2089.
  16. Schunemann HJ, Mustafa R, Brozek J, Santesso N, Alonso-Coello P, Guyatt G, et al. GRADE Guidelines: 16. GRADE evidence to decision frameworks for tests in clinical practice and public health. Journal of clinical epidemiology. 2016;76:89-98.
  17. Andrews JC, Schunemann HJ, Oxman AD, Pottie K, Meerpohl JJ, Coello PA, et al. GRADE guidelines: 15. Going from evidence to recommendation-determinants of a recommendation's direction and strength. Journal of clinical epidemiology. 2013;66:726-35.
  18. Andrews J, Guyatt G, Oxman AD, Alderson P, Dahm P, Falck-Ytter Y, et al. GRADE guidelines: 14. Going from evidence to recommendations: the significance and presentation of recommendations. Journal of clinical epidemiology. 2013;66:719-25.

In order to more efficiently deal with administrative issues, the GRADE Guidance Group (G3) was established in 2013 with a committee of up to 12 members (increased from 10 to 12 in 2020) including the two co-chairs, with a rotating membership, meeting bimonthly.

Their role is to:

  1. Act as an executive committee for the GRADE Working Group (GWG)
  2. Provide guidance and ensure accountability of the GWG co-chairs
  3. Help set GRADE policies and directions
  4. Report to GRADE working group at each GRADE meeting