How to Use the SIGN Methodology Checklist for RCTs

The Scottish Intercollegiate Guidelines Network (SIGN) provides a structured checklist to assess the methodological quality of randomized controlled trials (RCTs). It helps determine whether an RCT’s findings are reliable and valid for clinical decision-making.

Step-by-Step Guide

1. Access the SIGN Checklist


2. Understand the Key Appraisal Domains

The checklist evaluates internal validity (risk of bias) and overall reliability of an RCT. Key sections include:

A. Internal Validity (Bias Risk Assessment)

  1. Randomization
    • Was the allocation sequence truly random?
    • Was allocation concealment adequate?
  2. Blinding (Masking)
    • Were participants, investigators, and outcome assessors blinded?
  3. Baseline Characteristics
    • Were groups similar at baseline?
  4. Follow-up & Attrition
    • Were all participants accounted for?
    • Was intention-to-treat (ITT) analysis used?
  5. Outcome Measurement
    • Were outcomes clearly defined and objectively measured?

B. Statistical & Clinical Relevance

  • Were statistical methods appropriate?
  • Are the results clinically meaningful?

3. Apply the Checklist to the RCT

For each question, answer: ✅ Yes (low risk of bias) ❌ No (high risk of bias) ❓ Unclear/Cannot say (insufficient information)

Example Assessment

Question Response Justification
Was the randomization method clearly described and appropriate? ✅ Yes Computer-generated random sequence used.
Was allocation concealment maintained? ❌ No No mention of sealed envelopes or central randomization.
Were participants and personnel blinded? ❓ Unclear States "double-blind" but does not specify methods.
Were outcome assessors blinded? ✅ Yes Independent assessors unaware of treatment allocation.
Were dropouts/loss to follow-up <20%? ✅ Yes Only 5% attrition, balanced between groups.
Was an intention-to-treat (ITT) analysis performed? ❌ No Per-protocol analysis used instead.

4. Assign an Overall Quality Rating

Based on responses, categorize the study as:
  • ++ (High Quality): Minimal bias, reliable results.
  • + (Acceptable Quality): Some weaknesses but unlikely to affect conclusions.
  • – (Low Quality): Significant flaws, high risk of bias.
Example:
  • Randomization: ✅
  • Blinding: ❓
  • Attrition: ✅
  • ITT Analysis: ❌ ➔ Overall Rating: + (Acceptable but with limitations)

5. Use the Appraisal in Evidence Synthesis

  • Include only high-quality (++) studies in strong recommendations.
  • Low-quality (–) studies may require cautious interpretation.

Key Tips for Effective Use

✔ Compare with other tools (e.g., Cochrane RoB 2, CASP) for cross-validation. ✔ Check for protocol deviations (e.g., ClinicalTrials.gov) to detect selective reporting. ✔ Use in systematic reviews or clinical guideline development. Would you like a real-paper example assessed using SIGN?
How to use Newcastle-Ottawa Scale (NOS)