Systematic Reviews: Reporting the quality/risk of bias

GRADE has two levels: strong and weak recommendations. It is a tool for judging the body of evidence as a whole.

The tool is structured into five domains through which bias might be introduced into the result. The evaluation is assessed into one of 3 categories: high risk of bias, some concerns, and low risk of bias. The assessment is specific to a single trial result that is an estimate of the relative effect of two interventions or intervention strategies on a particular outcome.

The tool has seven domains for appraising non-randomized observational studies such as cohort studies and case-control studies in which intervention groups are allocated during the course of usual treatment decisions and subsequently lack detail, or quasi-randomised studies in which the method of allocation falls short of full randomisation. The evaluation is expressed in terms of 5 outcomes from Low risk of bias to Critical risk bias, or No information on which to base a judgement.

Grades of Recommendation are A-D based on a table evaluating levels of evidence for five types of review: Therapy/Prevention, Aetiology/Harm; Prognosis; Diagnosis; Differential diagnosis/symptom prevalence study; and Economic and decision analyses

The higher the score, the higher the quality, this permits an author to rank the studies in a review. The scale was introduced in the journal article Jadad, A., Moore, R., Carroll, D., Jenkinson, C., Reynolds, D., Gavaghan, D., & McQuay, H. (1996). Assessing the quality of reports of randomized clinical trials: is blinding necessary?. Controlled Clinical Trials, 17(1), 1-12.

In the Newcastle-Ottawa Scale (NOS) the reviewer assigns a star rating to case-control studies in three areas: selection, comparability, and exposure.

Unlike the methods described above this checklist does not compute a score but rather can be used to ensure assumptions and limitations are understood and taken into account when interpreting validity and generalizability. Risk of Bias assessment for AHRQ Evidence-based Practice Centers (EPCs) for assessing the quality of studies included in Comparative Effectiveness reviews.

This checklist does not compute a score but rather can be used to ensure assumptions and limitations are understood and taken into account when interpreting validity and generalizability. See the justifying article at http://journals.cambridge.org/action/displayFulltext?type=1&pdftype=1&fid=292675&jid=THC&volumeId=21&issueId=02&aid=292673

QUADAS is for assessing the quality of diagnostic accuracy studies. Use this tool when you are following the STARD guidelines for systematic reviews of diagnostic studies.

AGREE is a tool for assessing the quality of Clinical Practice Guidelines. It is not to be used to appraise journal articles reporting the results of clinical trials but instead it is a quality assessment instrument for evaluating or deciding which guidelines could be recommended for use in practice or to inform health policy decisions.

A 15-item instrument using scales of 1-5 to evaluate a guideline's adherence to the Institute of Medicine's standard for trustworthy guidelines. It has good external validity among guideline developers and good interrater reliability across trained reviewers.

A method for evaluating the evidence about environmental contaminants and their potential effects on reproductive and developmental health. The GRADE method considers only human experimental and observational evidence. The Navigation Guide also rates studies of laboratory animals and other nonhuman streams of evidence. The result is one of five possible statements about the overall strength of the evidence pertaining to a particular environmental exposure: “known to be toxic,” “probably toxic,” “possibly toxic,” “not classifiable,” or “probably not toxic” to reproductive or developmental health. The link is to the appendix of Woodruff, T., & Sutton, P. (2011). An Evidence-Based Medicine Methodology To Bridge The Gap Between Clinical And Environmental Health Sciences. Health Affairs, 30(5), 931-937.

NutriGrade is for evaluating the quality of evidence in Human Nutrition research

From the CLARITY Group at McMaster University, includes:
Tool to Assess Risk of Bias in Cohort Studies
Tool to Assess Risk of Bias in Case Control Studies
Tool to Assess Risk of Bias in Randomized Controlled Trials
Tool to Assess Risk of Bias in Longitudinal Symptom Research Studies Aimed at the General Population
Risk of bias instrument for cross-sectional surveys of attitudes and practices.

Used to evaluate the quality of individual Systematic Reviews, Diagnostic, Prognosis, Randomised Controlled Trials (RCT), or Qualitative Studies

Does not compute a score, but could be used to double check the content validity of analytical reviews of randomized studies. AMSTAR is an 11-item checklist for evaluating the methodological quality of systematic reviews of randomized/RCT's. The equivalent tool for examining systematic reviews of non-randomized/observational studies would be RoBANS.

Does not compute a score but could be used when examining systematic reviews of non-randomized/observational studies. The RoBANS tool looks at six domains of study methodology: the selection of participants, confounding variables, the measurement of exposure, the blinding of the outcome assessments, incomplete outcome data, and selective outcome reporting. The equivalent tool for examining systematic reviews of randomized/RCT's would be AMSTAR.

ROBIS is a tool for assessing the risk of bias in systematic reviews (rather than in primary studies). The target audience of ROBIS is primarily guideline developers, and authors of overviews of systematic reviews (‘‘reviews of reviews’’).