Skip to Main Content
It looks like you're using Internet Explorer 11 or older. This website works best with modern browsers such as the latest versions of Chrome, Firefox, Safari, and Edge. If you continue with this browser, you may see unexpected results.
Apply a grading criteria to your selected studies: PRISMA Item 12
When reading the full text of each article identified for inclusion in the review you may wish to apply one of the following Scales/Assessment for quality to each study selected for inclusion: you can choose a method that best fits with your type of review, but before you make your selection first please read other reviews written by/for subject matter experts in your discipline/field/profession, you want to use a grading criteria recognized and used by your peers. You can report the quality/risk of bias scale you used in your Methods section, and report the grade/level of quality you assign to each study either summarised in the results section or as an extra column in your study characteristics table.
GRADE - Grading of Recommendations Assessment, Development and Evaluation
GRADE has two levels: strong and weak recommendations. It is a tool for judging the body of evidence as a whole.
RoB 2: Cochrane risk-of-bias tool for randomized trials
The tool is structured into five domains through which bias might be introduced into the result. The evaluation is assessed into one of 3 categories: high risk of bias, some concerns, and low risk of bias. The assessment is specific to a single trial result that is an estimate of the relative effect of two interventions or intervention strategies on a particular outcome.
ROBINS-I: Cochrane risk-of-bias tool for non-randomised studies of interventions
The tool has seven domains for appraising non-randomized observational studies such as cohort studies and case-control studies in which intervention groups are allocated during the course of usual treatment decisions and subsequently lack detail, or quasi-randomised studies in which the method of allocation falls short of full randomisation. The evaluation is expressed in terms of 5 outcomes from Low risk of bias to Critical risk bias, or No information on which to base a judgement.
CEBM Levels of Evidence, Oxford Centre for Evidence Based Medicine
Grades of Recommendation are A-D based on a table evaluating levels of evidence for five types of review: Therapy/Prevention, Aetiology/Harm; Prognosis; Diagnosis; Differential diagnosis/symptom prevalence study; and Economic and decision analyses
JADAD - Jadad quality assessment scale for rating Randomized Controlled Trials
The higher the score, the higher the quality, this permits an author to rank the studies in a review. The scale was introduced in the journal article Jadad, A., Moore, R., Carroll, D., Jenkinson, C., Reynolds, D., Gavaghan, D., & McQuay, H. (1996). Assessing the quality of reports of randomized clinical trials: is blinding necessary?. Controlled Clinical Trials, 17(1), 1-12.
NOS - Newcastle-Ottawa quality assessment scale for case-control studies
In the Newcastle-Ottawa Scale (NOS) the reviewer assigns a star rating to case-control studies in three areas: selection, comparability, and exposure.
AHRQ checklist for Risk of Bias assessment in Comparative Effectiveness Reviews
Unlike the methods described above this checklist does not compute a score but rather can be used to ensure assumptions and limitations are understood and taken into account when interpreting validity and generalizability. Risk of Bias assessment for AHRQ Evidence-based Practice Centers (EPCs) for assessing the quality of studies included in Comparative Effectiveness reviews.
CHEC-list for methodological quality in Cost Effectiveness Analyses & Comparative Effectiveness Reviews
This checklist does not compute a score but rather can be used to ensure assumptions and limitations are understood and taken into account when interpreting validity and generalizability. See the justifying article at http://journals.cambridge.org/action/displayFulltext?type=1&pdftype=1&fid=292675&jid=THC&volumeId=21&issueId=02&aid=292673
QUADAS-2 QUality Assessment tool for Diagnostic Accuracy Studies
QUADAS is for assessing the quality of diagnostic accuracy studies. Use this tool when you are following the STARD guidelines for systematic reviews of diagnostic studies.
AGREE-II Appraisal of Guidelines for Research and Evaluation
AGREE is a tool for assessing the quality of Clinical Practice Guidelines. It is not to be used to appraise journal articles reporting the results of clinical trials but instead it is a quality assessment instrument for evaluating or deciding which guidelines could be recommended for use in practice or to inform health policy decisions.
National Guideline Clearinghouse Extent of Adherence to Trustworthy Standards (NEATS) Instrument
A 15-item instrument using scales of 1-5 to evaluate a guideline's adherence to the Institute of Medicine's standard for trustworthy guidelines. It has good external validity among guideline developers and good interrater reliability across trained reviewers.
The Navigation Guide: Environmental Health/Toxicology studies
A method for evaluating the evidence about environmental contaminants and their potential effects on reproductive and developmental health. The GRADE method considers only human experimental and observational evidence. The Navigation Guide also rates studies of laboratory animals and other nonhuman streams of evidence. The result is one of five possible statements about the overall strength of the evidence pertaining to a particular environmental exposure: “known to be toxic,” “probably toxic,” “possibly toxic,” “not classifiable,” or “probably not toxic” to reproductive or developmental health. The link is to the appendix of Woodruff, T., & Sutton, P. (2011). An Evidence-Based Medicine Methodology To Bridge The Gap Between Clinical And Environmental Health Sciences. Health Affairs, 30(5), 931-937.
Risk of Bias assessment instruments
From the CLARITY Group at McMaster University, includes:
Tool to Assess Risk of Bias in Cohort Studies
Tool to Assess Risk of Bias in Case Control Studies
Tool to Assess Risk of Bias in Randomized Controlled Trials
Tool to Assess Risk of Bias in Longitudinal Symptom Research Studies Aimed at the General Population
Risk of bias instrument for cross-sectional surveys of attitudes and practices.
Critical Appraisal Worksheets Oxford Centre for Evidence Based Medicine (CEBM)
Used to evaluate the quality of individual Systematic Reviews, Diagnostic, Prognosis, Randomised Controlled Trials (RCT), or Qualitative Studies
Evaluating and Critically Appraising Systematic Reviews
AMSTAR: a measurement tool to assess the methodological quality of systematic reviews
Does not compute a score, but could be used to double check the content validity of analytical reviews of randomized studies. AMSTAR is an 11-item checklist for evaluating the methodological quality of systematic reviews of randomized/RCT's. The equivalent tool for examining systematic reviews of non-randomized/observational studies would be RoBANS.
RoBANS: Risk of Bias Assessment Tool for Nonrandomized Studies
Does not compute a score but could be used when examining systematic reviews of non-randomized/observational studies. The RoBANS tool looks at six domains of study methodology: the selection of participants, confounding variables, the measurement of exposure, the blinding of the outcome assessments, incomplete outcome data, and selective outcome reporting. The equivalent tool for examining systematic reviews of randomized/RCT's would be AMSTAR.
ROBIS is a tool for assessing the risk of bias in systematic reviews (rather than in primary studies). The target audience of ROBIS is primarily guideline developers, and authors of overviews of systematic reviews (‘‘reviews of reviews’’).