JBI Critical Appraisal Tools: A Practical Guide

The Joanna Briggs Institute maintains its own family of critical appraisal checklists, tailored to specific study designs, reflecting JBI's broader native support for review types beyond intervention effectiveness alone. Understanding which checklist applies to which design, and how the appraisal process actually works, matters for anyone following JBI methodology.

Why JBI uses design-specific checklists

Rather than a small number of broadly applicable tools, JBI provides separate checklists for randomized controlled trials, quasi-experimental studies, cohort studies, case-control studies, analytical cross-sectional studies, case reports, case series, qualitative research, systematic reviews themselves, diagnostic test accuracy studies, prevalence studies, and economic evaluations. Each checklist reflects the specific bias concerns relevant to that design, rather than applying a one-size-fits-all set of questions across genuinely different study types.

The RCT checklist

The JBI checklist for randomized controlled trials covers domains including true randomization, allocation concealment, baseline comparability between groups, blinding of participants and assessors, completeness of follow-up, and appropriateness of the statistical analysis -- conceptually overlapping with RoB 2's domains, though structured as a checklist of yes/no/unclear/not-applicable questions rather than RoB 2's more elaborated signaling-question and algorithm-based approach.

The cohort study checklist

For cohort studies, the JBI checklist addresses whether exposed and unexposed groups were recruited from the same population, whether exposure was measured validly, whether confounding factors were identified and addressed, and whether follow-up was complete and outcomes assessed validly -- domains reflecting the specific bias concerns relevant to observational, non-randomized comparison.

The qualitative research checklist

JBI's qualitative appraisal checklist addresses congruity between the stated philosophical perspective and the research methodology, between the methodology and the research question, and between the methodology and methods of data collection and analysis, alongside questions about researcher reflexivity and ethical approval -- reflecting qualitative research's different epistemological foundations compared to quantitative study appraisal.

How the scoring works in practice

Each checklist item is typically scored as yes, no, unclear, or not applicable, without a single mechanically computed overall score in most versions -- JBI methodology generally expects reviewers to use the checklist to inform an overall judgment about a study's methodological quality and relevance for inclusion, rather than applying a fixed numeric cutoff. This holistic judgment approach is a genuine methodological choice, not a gap in the tool, though it does place more weight on reviewer training and inter-rater consistency than a more mechanically scored tool would.

Dual appraisal and disagreement

As with screening and extraction, JBI methodology expects two independent reviewers to conduct critical appraisal, with disagreements resolved through discussion or a third reviewer, and the process documented transparently in your methods section.

Appraisal and inclusion decisions

JBI methodology generally does not use critical appraisal scores as a strict inclusion or exclusion filter on their own -- a lower-quality study is not automatically excluded, but its limitations should inform how much weight it carries in your synthesis and how you discuss it in your findings. This differs from some other approaches that use a quality threshold as a hard eligibility criterion.

Choosing the right checklist for a mixed-design review

A JBI-methodology review including multiple study designs needs to apply the correct checklist to each included study individually, matched to its actual design -- applying the RCT checklist to an observational cohort study, or vice versa, is a mismatch a methods-aware reviewer will notice immediately, exactly parallel to the RoB 2 versus ROBINS-I mismatch that draws the same kind of objection in Cochrane-methodology reviews.

A practical starting point

Before beginning appraisal, confirm you have identified the correct JBI checklist for every included study design in your review, and that whoever is conducting appraisal has genuine familiarity with that specific checklist's questions and scoring conventions -- this upfront preparation prevents the more time-consuming problem of discovering a mismatched or inconsistently applied checklist partway through the appraisal process.

Training reviewers on JBI checklists before appraisal begins

Because JBI's holistic scoring approach relies more heavily on reviewer judgment than a more mechanically scored tool, investing time in aligning your appraisal team's understanding of each checklist item before starting formal appraisal genuinely improves inter-rater consistency. A short calibration exercise -- both reviewers independently appraising the same one or two studies and then discussing any disagreement -- surfaces differences in interpretation before they propagate across your full study list.

Where to find the current checklists

JBI periodically updates its appraisal checklists, and using an outdated version can introduce subtle inconsistencies with current JBI methodology guidance. Downloading checklists directly from JBI's own manual and resources rather than from a secondary source or a template inherited from a previous project ensures you're working from the current, correctly worded version, particularly important given how much the holistic scoring approach depends on the exact wording of each checklist item.

JBI appraisal and your eventual synthesis narrative

Because JBI methodology doesn't use appraisal scores as a strict inclusion filter, your synthesis and discussion sections need to do the work of communicating how appraisal findings actually shaped your confidence in specific results -- explicitly noting, for instance, that a particular finding rests primarily on studies appraised as lower quality, rather than letting appraisal results sit in a supplementary table disconnected from how confidently your narrative presents its conclusions. This link between appraisal and narrative confidence is easy to lose sight of once appraisal itself is technically complete, which is exactly why it deserves a deliberate check during the writing stage, not just during appraisal itself. Building in a dedicated review pass specifically focused on this connection, after your synthesis narrative is otherwise drafted, catches the disconnect between appraisal findings and narrative confidence far more reliably than hoping it gets addressed naturally while writing under normal drafting pressure. Scheduling it as its own distinct step, with its own checklist, rather than trusting it will happen naturally alongside everything else competing for attention during a manuscript's final drafting stages, is what actually makes this connection reliable rather than aspirational. Treating it as a genuine checklist item, not an assumed byproduct of careful writing, is what actually makes the difference in practice.