Critical appraisal is usually unnecessary in a scoping review because the aim is to map the existing literature rather than assess, rank or weigh the quality of evidence. A critical appraisal is more appropriate when the scoping review specifically investigates methodological quality or has another clear reason for evaluative judgement.

What is a critical appraisal?

A critical appraisal is a structured assessment of how trustworthy and methodologically sound a study is. It uses a checklist or tool, such as CASP, JBI or the Cochrane Risk of Bias tools, to assess issues including bias, study design, sampling, ethics, data collection and the reliability of conclusions.

A critical appraisal is therefore concerned with how well a study was conducted, rather than simply what the study found.

It is normally used for one of two purposes:

1. To decide whether a study should be included or excluded.

2. To give more weight to some studies than others when drawing conclusions.

The purpose of your review matters because these functions involve evaluating the evidence, while a scoping review is generally designed to map the available literature.

Why should you generally avoid critical appraisal in a scoping review?

There is usually no need to conduct a critical appraisal because you do not weigh or rank evidence in a scoping review. The aim is to provide an overview or map of the existing literature, which means including the full range of research on the topic regardless of quality.

This applies across different postgraduate research areas. Education students may examine curriculum design, learner outcomes or programme structures, while healthcare students may examine intervention designs, population characteristics or outcome measures. In these cases, research quality is often not the primary interest.

Critical appraisal can also create practical and methodological problems. Excluding lower-quality studies can restrict the landscape you are supposed to map and produce an incomplete or distorted representation of the field.

A scoping review may also include peer-reviewed studies, reports, policy documents, white papers, curriculum documents and other grey literature. Appraisal tools do not exist for every type of source, creating an inconsistent process.

Can critical appraisal confuse a scoping review with a systematic review?

Critical appraisal can make the identity and purpose of your review less clear. One of the differences between a scoping review and a systematic review is the presence of quality appraisal.

If your scoping review is already close to a systematic review, such as when exploring barriers and facilitators or synthesising patient experiences, adding critical appraisal can raise the question of why you did not conduct a systematic review instead.

There are also practical costs. Critical appraisal is time-consuming, adds substantial workload and uses valuable word count that could otherwise be spent on analysis, synthesis and discussion.

A critical appraisal increases rigour when the purpose of the review requires selectivity or evaluative judgement. Those are not appropriate aims for most scoping reviews.

If you cannot articulate a strong academic rationale for including critical appraisal, adding it simply to create another section in your dissertation can become an inappropriate methodological choice.

When should you include critical appraisal in a scoping review?

There are circumstances where critical appraisal is appropriate. In particular, it can be useful when the scoping review itself is interested in methodology and its quality.

For example, one scoping review examined nurse-led randomised controlled trials and used a critical appraisal tool because the review was specifically interested in the methodological quality of those trials. The tool was not used to exclude papers.

Another example involved a scoping review of systematic reviews about research integrity. The topic itself was directly related to quality appraisal, making critical appraisal relevant to the purpose of the review.

The important point is that the appraisal needs to serve the purpose of the scoping review. It should not be included simply because guidance describes critical appraisal as optional or because you want to add another methodological component.

How should you decide whether your scoping review needs critical appraisal?

Most scoping reviews do not need a critical appraisal. The starting point should be the purpose of your review and what you are trying to achieve by mapping the literature.

Ask whether assessing methodological quality is actually part of your research interest. If your aim is simply to understand the range, characteristics and landscape of available research, critical appraisal is generally unnecessary.

If methodological quality itself is an object of interest, there may be a stronger rationale for including it. In that situation, the appraisal should be clearly connected to your research purpose rather than added as an isolated methodological step.

Keeping your focus on mapping the landscape and staying true to the purpose of the review will help you make a more defensible methodological decision.