ROBIS (Risk Of Bias In Systematic Reviews) is a structured tool designed to assess the risk of bias in systematic reviews (SRs) across various domains. Unlike AMSTAR-2 (which evaluates methodological quality), ROBIS specifically focuses on identifying biases that could distort the review’s conclusions.

ROBIS: Step-by-Step Guide

1. Understand the 3 Phases of ROBIS

ROBIS is applied in three phases:
Phase Purpose
Phase 1 Assess relevance (optional) – Is the review relevant to your research question?
Phase 2 Identify concerns about bias in 4 key domains
Phase 3 Judge overall risk of bias in the review

2. Phase 2: Assess Bias in 4 Key Domains

Each domain is rated as "Low," "High," or "Unclear" risk of bias.

Domain 1: Study Eligibility Criteria

  • Did the review use appropriate inclusion/exclusion criteria?
  • Were the criteria pre-specified (e.g., in a protocol)?
  • Bias risk: High if criteria were modified after seeing results.

Domain 2: Identification and Selection of Studies

  • Was the search strategy comprehensive (databases, grey literature, no language restrictions)?
  • Were multiple reviewers involved in study selection?
  • Bias risk: High if key studies were missed due to poor search methods.

Domain 3: Data Collection and Study Appraisal

  • Were data extracted independently by multiple reviewers?
  • Was risk of bias assessed in included studies (e.g., using Cochrane RoB tool)?
  • Bias risk: High if subjective decisions influenced results.

Domain 4: Synthesis and Findings

  • Were appropriate methods used for meta-analysis (if applicable)?
  • Was heterogeneity explored?
  • Were conflicts of interest reported?
  • Bias risk: High if synthesis methods were flawed (e.g., selective reporting).

3. Phase 3: Final Judgment of Overall Risk of Bias

After assessing all domains, determine the overall risk of bias:
  • Low risk = No major concerns in any domain.
  • High risk = Serious concerns in ≥1 domain.
  • Unclear risk = Insufficient information.

When to Use ROBIS?

✔ Critical appraisal of SRs for clinical guidelines. ✔ Peer review of manuscripts. ✔ Comparing multiple SRs on the same topic.

Example ROBIS Assessment

Review Topic: "Effectiveness of probiotics for antibiotic-associated diarrhea."
Domain Judgment Reason
Eligibility Criteria Low risk Clearly defined PICO criteria.
Study Selection High risk Only English-language studies included.
Data Collection Low risk Dual extraction & RoB assessment done.
Synthesis Unclear risk Heterogeneity not fully explored.
Overall Risk of Bias High Language bias could skew results.

Strengths of ROBIS

✔ Focuses specifically on bias (unlike AMSTAR-2, which covers general quality). ✔ Applicable to all SRs (including non-intervention studies). ✔ Clear structured approach with signaling questions.

Limitations

✖ Subjective judgments required (experience helps). ✖ Time-consuming compared to simpler tools like CASP.

ROBIS vs. AMSTAR-2

Feature ROBIS AMSTAR-2
Focus Bias assessment Methodological quality
Scoring Low/High/Unclear risk High/Moderate/Low/Critically Low confidence
Best for Detecting flawed conclusions Overall quality check

Conclusion

ROBIS is a powerful tool for detecting bias in systematic reviews, helping clinicians and researchers decide whether to trust a review’s findings.