Multi-Standards Checklist Engine
PRISMA 2020 is the baseline, but a real review may need to satisfy several reporting standards simultaneously — PRISMA-S for search reproducibility, ROBIS for review-level bias, AMSTAR-2 for methodological quality, and agency-specific checklists on top. EvidAI's checklist engine evaluates every applicable standard automatically and ships each one as a file in the export bundle.
Supported Standards
| Standard | Applies to |
|---|---|
| PRISMA 2020 — 27-item reporting checklist | All systematic reviews |
| PRISMA-S — search reporting extension | Search phase |
| PROSPERO — protocol registration checklist | Protocol phase |
| ROBIS — Risk of Bias in Systematic Reviews | Overall review bias |
| AMSTAR-2 — 16-item critical appraisal of SRs | Overall review quality |
| RAISE — Reviewer AI and Statistical Evaluation | AI-assisted reviews |
| Cochrane MECIR — conduct + reporting standards | Cochrane reviews |
| JBI Critical Appraisal | JBI methodology reviews |
| STROBE — observational study reporting | Included observational studies |
| CONSORT 2010 — RCT reporting | Included RCTs |
| NICE UK HTA submission | HTA reporting profile |
| CADTH reimbursement submission | HTA reporting profile |
| IQWiG benefit assessment dossier | HTA reporting profile |
| ICH M4E(R2) Module 2.5 Clinical Overview | ICH reporting profile |
The registry is extensible — additional standards can be added without changing the consuming code paths.
How Evaluation Works
Each checklist is a structured definition containing a label, version, and list of items. Each item knows which review-state signals it depends on and emits a status of complete, partial, or missing for the active review.
The evaluator takes the review's phase signals, the artifacts already present in the export bundle, and the active reporting profile, then walks every active checklist deterministically. Missing signals degrade gracefully — a missing input produces a missing status for the affected item, not an error.
Profile-Driven Activation
Which checklists are active is driven by the review's Reporting Standard profile:
| Reporting profile | Active checklists |
|---|---|
| PRISMA 2020 (default) | prisma-2020, prisma-s, prospero, robis, amstar-2 |
| Cochrane intervention | prisma-2020, prisma-s, cochrane-mecir, robis, amstar-2 |
| JBI | prisma-2020, jbi-sumari, robis |
| HTA dossier | prisma-2020, prisma-s, nice-hta, cadth-hta, iqwig-hta |
| ICH M4E + EMA RWE | prisma-2020, ich-m4e |
| AI-reported (RAISE) | prisma-2020, raise |
The checklist engine iterates over the profile's active standards on every export.
Export Bundle Artifacts
Every bundle emits one Markdown + one JSON + one CSV per active checklist (e.g. PRISMA 2020 ships as a human-readable Markdown report, a machine-readable JSON, and a spreadsheet-friendly CSV — and the same triple is produced for every other active standard). A README in the checklists directory indexes every attached checklist and its coverage score.
Quality Thresholds as Rules
In addition to item-level status, the engine evaluates phase-level quality thresholds that you can set per review:
- Minimum inter-rater agreement (Cohen's Kappa) for dual screening
- Minimum proportion of papers dual-extracted
- Maximum proportion of high-risk-of-bias studies
A threshold fail degrades the containing checklist item (typically to partial) and the reason is surfaced in the bundle's Markdown output.
How to Configure
Set the reporting profile
- SLR hub → Reporting Standard selector → pick a profile.
- The profile is saved against the review with signer and timestamp, and triggers a Reporting profile selected audit event.
Adjust quality thresholds
- SLR hub → Options drawer on any phase card → Quality thresholds section.
- Edit the numeric threshold and save.
Preview checklist state
The Reporting Standard selector shows a live coverage score — how many items are complete versus missing across all active checklists. Clicking it opens a per-item breakdown.
Design Principles
- Pure and deterministic — given the same review state, the engine always returns the same status. This makes the bundle output reproducible.
- Fail-open — missing signals never throw; they mark the relevant item as missing and keep the rest of the checklist evaluating.
- Profile-gated — checklists are only evaluated if the review's active profile lists them. Irrelevant standards never clutter the bundle.