AMSTAR (A MeaSurement Tool to Assess Systematic Reviews)

AMSTAR (A MeaSurement Tool to Assess Systematic Reviews) is a widely used critical appraisal tool designed to evaluate the methodological quality of systematic reviews (SRs), including meta-analyses. The original AMSTAR-1 (2007) was updated to AMSTAR-2 (2017) to improve its applicability to both randomized and non-randomized studies of healthcare interventions.

Key Features of AMSTAR-2

AMSTAR-2 consists of 16 items (compared to 11 in AMSTAR-1), assessing different aspects of a systematic review’s methodology. Unlike the original version, AMSTAR-2 does not provide a numerical score but instead evaluates confidence in the review’s results based on critical weaknesses.

Domains Assessed by AMSTAR-2

  1. Protocol Registration
    • Was a review protocol registered before starting the study? (Reduces bias from post-hoc changes).
  2. Comprehensive Literature Search
    • Did the review use multiple databases, grey literature, and manual searches?
  3. Justification for Study Selection
    • Were inclusion/exclusion criteria clearly defined?
  4. Duplicate Study Selection & Data Extraction
    • Were screening and data extraction performed by at least two reviewers independently?
  5. Risk of Bias (RoB) Assessment in Individual Studies
    • Did the review assess RoB (e.g., using Cochrane’s RoB tool, Newcastle-Ottawa Scale)?
  6. Appropriate Meta-Analysis Methods
    • Were statistical methods (e.g., fixed/random effects) justified?
  7. Consideration of RoB in Synthesis
    • Did the review account for RoB when interpreting results?
  8. Publication Bias Assessment
    • Was a test for publication bias conducted (e.g., funnel plot, Egger’s test)?
  9. Conflict of Interest (COI) Reporting
    • Were funding sources and author COIs disclosed?
(Additional items cover heterogeneity, subgroup analysis, and more.)

AMSTAR-2 Rating System

Instead of a numerical score, AMSTAR-2 classifies confidence in the review’s findings as:
Rating Description
High No or one non-critical weakness.
Moderate More than one non-critical weakness.
Low One critical flaw (e.g., no RoB assessment).
Critically Low Multiple critical flaws.

Strengths of AMSTAR-2

✔ Broad Applicability: Works for both randomized and non-randomized reviews. ✔ Detailed Assessment: Covers all key methodological aspects. ✔ No Overemphasis on Scoring: Prevents misleading "quality scores."

Limitations

✖ Subjective Judgments: Some items require interpretation. ✖ Not a Reporting Guideline: Unlike PRISMA, it assesses methodology, not just reporting.

When to Use AMSTAR-2?

  • Evaluating SRs for clinical guidelines.
  • Peer-reviewing manuscripts.
  • Conducting umbrella reviews (reviews of reviews).

Example AMSTAR-2 Checklist in Practice

If a review: ✅ Registered a protocol (Item 2) ✅ Used multiple databases (Item 4) ❌ Did not assess publication bias (Critical flaw) → It would likely be rated "Critically Low" confidence.

Conclusion

AMSTAR-2 is the gold standard for assessing systematic review quality. While it doesn’t generate a score, its structured approach helps identify flaws that could undermine a review’s reliability. Would you like a downloadable AMSTAR-2 checklist or guidance on applying it to a specific review?