AMSTAR-2: Appraising the Quality of Systematic Reviews

If your project involves an umbrella review, or any task that requires judging whether an existing systematic review was conducted rigorously, AMSTAR-2 is the tool built for exactly that job. It's often confused with the original AMSTAR, which it substantially expanded and revised, particularly around risk-of-bias handling in the included primary studies.

The structure: 16 items, 7 critical domains

AMSTAR-2 contains 16 items, each rated as Yes, Partial Yes, or No. Seven of these are designated critical domains, and your overall confidence rating for a review depends heavily on how it performs specifically on these seven, not just on the total item count:

1. Did the research questions and inclusion criteria include PICO components? 2. Was a comprehensive literature search conducted? 3. Was study selection performed in duplicate? 4. Was data extraction performed in duplicate? 5. Was a list of excluded studies provided, with reasons? 6. Were the risk-of-bias assessments of individual included studies adequate? 7. Was risk of bias accounted for when interpreting the review's results?

A review can score reasonably well on the full 16-item count while still earning a low overall confidence rating if it falls short on one or more of these seven, which is precisely the design intent. AMSTAR-2 doesn't reward volume of good practice; it penalizes weaknesses in the domains judged to matter most for trustworthiness.

Overall confidence ratings

Based on the pattern of critical and non-critical weaknesses, AMSTAR-2 assigns one of four overall confidence ratings to the appraised review:

- **High:** no or one non-critical weakness - **Moderate:** more than one non-critical weakness - **Low:** one critical flaw, with or without non-critical weaknesses - **Critically low:** more than one critical flaw

This is a genuinely different logic from a simple percentage score, and it's the part most first-time users get wrong: a review with a single critical flaw in, say, inadequate risk-of-bias assessment cannot score higher than "Low" overall, no matter how strong the rest of its methodology is.

Where AMSTAR-2 fits in your workflow

**Umbrella reviews.** If you're synthesizing across multiple existing systematic reviews, AMSTAR-2 is the standard tool for weighting how much confidence to place in each included review, and its ratings should directly inform how you present and interpret conflicting findings across reviews.

**Choosing which reviews to cite or build on.** Before treating an existing systematic review as a reliable foundation for further work, a quick AMSTAR-2 appraisal tells you whether its conclusions are well-supported or should be treated cautiously.

**Peer review and editorial screening.** Some journals now use AMSTAR-2 internally to triage submitted systematic reviews before sending them out for full peer review, which means understanding it also helps you anticipate where your own review might draw scrutiny.

Common mistakes when applying it

**Treating all 16 items as equally weighted.** As above, the seven critical domains carry disproportionate weight in the overall rating. Missing this distinction leads to confidently wrong overall ratings.

**Applying it to primary studies instead of reviews.** AMSTAR-2 appraises systematic reviews themselves, not the individual primary studies inside them. That's a different task, requiring RoB 2, ROBINS-I, or another study-level tool instead.

**Skipping duplicate appraisal.** As with other appraisal tools, AMSTAR-2 should be applied by at least two independent reviewers, since items like adequacy of risk-of-bias handling require genuine judgment calls that benefit from a second opinion.

**[Get a quote](/systematic-reviews#quality-appraisal)** for AMSTAR-2 appraisal support on your umbrella review or evidence synthesis project.