The Newcastle-Ottawa Scale (NOS) is primarily designed for non-randomized studies (e.g., cohort and case-control studies), but it can be adapted for experimental studies (e.g., quasi-experimental or non-randomized trials). Below is a step-by-step guide on how to use it effectively.

1. Understand the NOS Structure (Adapted for Experimental Studies)

The NOS assesses studies based on three domains, each with sub-questions rated by stars (★). The maximum score is 9★ (highest quality).
Domain Key Criteria (Adapted for Experimental Studies)
Selection (4★ max) - Representativeness of exposed cohort - Selection of non-exposed group - Ascertainment of exposure - Demonstration that outcome was not present at start
Comparability (2★ max) - Controls for key confounders (e.g., age, sex, baseline differences) - Additional adjustments (if applicable)
Outcome (3★ max) - Independent/objective outcome assessment - Adequate follow-up duration - Completeness of follow-up

2. Step-by-Step Appraisal Process

A. Selection (Max 4★)

  1. Representativeness of the Exposed Group (1★)
    • ★ If the sample represents the target population (e.g., similar demographics, disease severity).
    • Example: "Patients were recruited from multiple clinics, reflecting real-world diversity."
  2. Selection of the Non-Exposed Group (1★)
    • ★ If the control group is drawn from the same source as the experimental group.
    • Example: "Controls were matched from the same hospital database."
  3. Ascertainment of Exposure (1★)
    • ★ If exposure (intervention) is reliably recorded (e.g., medical records, standardized protocols).
    • Example: "Treatment allocation was documented in trial logs."
  4. Demonstration That Outcome Was Not Present at Baseline (1★)
    • ★ If the study confirms no outcome differences between groups at baseline.
    • Example: "Baseline scores were similar (p > 0.05)."

B. Comparability (Max 2★)

  • Controls for Confounders (1–2★)
    • ★ If adjusted for key variables (e.g., age, sex, comorbidities).
    • ★★ If adjusted for multiple major confounders.
    • Example: "Analysis controlled for age, BMI, and baseline severity."

C. Outcome (Max 3★)

  1. Independent/Blinded Outcome Assessment (1★)
    • ★ If outcomes were assessed blindly or objectively.
    • Example: "Outcomes were evaluated by an independent blinded assessor."
  2. Adequate Follow-Up Duration (1★)
    • ★ If follow-up was long enough for outcomes to occur.
    • Example: "6-month follow-up was sufficient for detecting treatment effects."
  3. Completeness of Follow-Up (1★)
    • ★ If ≥80% follow-up rate or losses explained.
    • Example: "90% completed follow-up; dropouts were random."

3. Scoring & Interpretation

  • Total Score = Selection (4★) + Comparability (2★) + Outcome (3★)
  • Interpretation:
    • ≥7★ = High quality
    • 5–6★ = Moderate quality
    • ≤4★ = Low quality

Example Assessment

Domain Criteria Stars (★)
Selection Representativeness, matched controls ★★★★ (4/4)
Comparability Adjusted for age, sex, severity ★★ (2/2)
Outcome Blinded assessment, 85% follow-up ★★★ (3/3)
Total 9★ (High Quality)

4. When to Use NOS for Experimental Studies

  • Quasi-experimental studies (non-randomized trials)
  • Before-after studies (if control group is present)
  • Comparative effectiveness research (real-world evidence)
Limitations:
  • Not ideal for RCTs (use Cochrane RoB 2 instead).
  • Subjectivity in star allocation (use multiple reviewers for consistency).

5. Comparison with Other Tools

Tool Best For Max Score
NOS (Adapted) Non-randomized/quasi-experimental 9★
Cochrane RoB 2 RCTs Low/Some/High risk
SIGN RCT Checklist RCTs (guideline development) ++/+/-

Final Tips

✔ Use alongside other tools (e.g., ROBINS-I for non-randomized studies). ✔ Pre-define scoring rules to reduce subjectivity. ✔ Report NOS scores in systematic reviews for transparency. Would you like a real-world example of an NOS appraisal for a quasi-experimental study?