A scoping review develops through repeated searching, refinement and methodological decisions rather than a fixed step-by-step process. Preliminary research can reveal whether your research question is feasible, while AI tools such as ASReview can help manage large numbers of papers during screening without replacing your own research judgement.

How Do You Develop a Research Idea for a Scoping Review?

Before deciding on a scoping review, I wanted to understand how researchers use AI in their published qualitative research. I was interested in where and how AI was being used, how researchers described that use, and how these approaches could potentially be applied in dissertations and PhDs with appropriate transparency, ethical justification and citation.

A quick search suggested that there had been relatively little comprehensive discussion of AI use in qualitative research, despite a rapid growth in the literature. This led me towards a scoping review.

At this stage, however, the research idea was still messy. I had a vague idea of what I wanted to investigate, but no clear structure. Research planning is not like baking, where you follow a sequence of steps and eventually arrive at a finished product. It is more like hammering a piece of steel into a sword: you work on different parts, step back, assess what you have, and then return to refine it.

Your research plan will often need several rounds of searching, testing and refinement before you arrive at a feasible study.

How Do You Develop the Review Question and Research Objectives?

The initial review question for my scoping review was: How is AI used in healthcare qualitative research studies?

The research objectives were:

1. To map the methodological characteristics of healthcare qualitative studies that use AI.

2. To map the healthcare topics that use AI in qualitative research methods.

3. To describe how AI is used across healthcare qualitative research.

4. To identify areas and explore reasons where AI is not used in healthcare qualitative research.

At this point, I was only laying out the skeleton of the review based on my current knowledge and what I thought needed researching. I did not yet have clear key terms, inclusion and exclusion criteria or a selection process.

The purpose of the preliminary research was to learn more about the topic and identify what kind of refinement the research required. Everything was tentative and would be refined through further searching.

What Can You Learn From a Preliminary Search?

My first search used terms relating to artificial intelligence, machine learning and natural language processing, alongside terms for qualitative research. I searched for qualitative research itself, but also specific qualitative methodologies such as grounded theory, ethnography, case study, phenomenology and narrative research.

I also searched terms such as interviews, focus groups and observations because abstracts do not always clearly identify the methodology being used.

The search produced far too many studies. I initially narrowed the search to 2019–2024 and focused on healthcare, but I was still left with around 20,000 papers. A PubMed search using AI and qualitative research MeSH terms produced around 2,000 papers.

I therefore needed to investigate the results rather than simply assuming that the review was feasible. I exported 22,000 abstracts into a CSV file, removed 161 duplicates and began screening with ASReview.

What Problems Can Preliminary Research Reveal?

The preliminary search revealed that many papers were actually about qualitative research concerning the acceptability of AI, rather than studies using AI within qualitative research methods.

I also found many studies using AI to screen health and medical notes to develop predictors for suicide risk and other mental health issues. These were appearing partly because I had included the term “interview” in my search.

At the same time, SCOPUS AI helped me identify four potentially relevant papers. Most were theoretical rather than empirical, but they pointed towards additional literature and ideas that I could investigate through reference chaining.

From this preliminary work, I identified several changes I needed to make:

1. Remove interviews and focus groups from the search terms.

2. Include neural networks and deep learning in the search terms.

3. Find further ways to narrow the search.

4. Investigate the apparent focus on using AI for qualitative analysis.

This is one of the reasons preliminary research matters. It can reveal that the assumptions behind your original research question do not match the literature you are actually going to encounter.

When Should You Consult Another Academic About Your Scoping Review?

After the preliminary search, it became clear that my original plan was difficult to manage. There were too many papers, and it was difficult to screen the corpus of literature as a single researcher.

I therefore discussed the problem with another academic. He suggested several possibilities.

First, he questioned whether a scoping review was the most appropriate review method. A narrative review without a systematic search could potentially allow greater reliance on reference chaining, although I was interested in the charting stage of a scoping review.

Second, he suggested searching research methods journals because relevant methodological discussions might not appear in the field-specific literature.

Third, he suggested narrowing the research further rather than continuing to search across all qualitative research.

This discussion helped me reconsider the boundaries of the review rather than simply trying to force the original search strategy to work.

How Can You Refine the Search and Inclusion Criteria?

Following the preliminary research and discussion with another academic, I refined the search terms. I retained artificial intelligence, machine learning and natural language processing and their abbreviations, while adding deep learning and neural networks.

I also decided to focus specifically on thematic analysis, influenced by the theoretical papers I had found discussing the use of AI for qualitative analysis.

The refined search produced 510 papers on SCOPUS and 204 on PubMed.

I then developed inclusion and exclusion criteria.

Inclusion criteria:

1. Qualitative research articles that use AI in thematic analysis.

2. Healthcare, medicine and nursing research.

3. Published between 2021–2024.

4. Peer-reviewed articles.

Exclusion criteria:

1. Papers that do not clearly indicate where AI was used in thematic analysis.

2. Literature review and bibliometric studies using thematic analysis.

3. Papers that use AI in sentiment analysis but not thematic analysis.

How Can AI Help You Screen Papers in a Scoping Review?

After combining the searches and removing duplicates, I had 714 papers. Removing 132 duplicates left 582 papers for screening.

These papers were entered into ASReview. I initially reviewed a random sample of 11 abstracts, of which two were relevant and nine were irrelevant. This gave the machine learning model information to begin prioritising the papers.

ASReview learns from the decisions you make. As you classify papers as relevant or irrelevant, it moves papers that are more likely to be relevant higher in the queue. This means you are more likely to encounter potentially relevant papers earlier, while less relevant papers are pushed further down.

I also set a stopping criterion of 50 consecutive irrelevant papers. Once I reached 50 papers in a row that were considered irrelevant, I stopped screening.

I screened 387 of the 582 papers, which was 66% of the total. I identified 83 relevant records at the screening stage, representing 14% of the papers. Using ASReview saved me from reading a further 195 abstracts, which I estimated would have taken around eight additional hours.

What Are the Limitations of Using ASReview?

ASReview helped reduce the amount of screening I needed to do, but it also raised methodological questions.

Because I did not review the remaining papers, I was left wondering whether there were additional relevant papers that I had missed. I plan to return to them after completing the scoping review to investigate this for my own knowledge.

The machine learning system also needs relevant papers to begin learning. If you have not identified any relevant papers through your initial manual screening, the system cannot be trained in the same way.

The stopping criterion is another issue. I used 50 consecutive irrelevant papers because my dataset contained 582 papers. Other systematic reviewers have used 100 consecutive irrelevant papers with much larger datasets. There is research proposing a process for determining stopping criteria, but I was unable to apply all of those steps to my own review.

This means that using AI to screen papers can reduce workload, but it does not remove the methodological decisions that the researcher needs to make.

Why Is Preliminary Research Important for a Scoping Review?

The biggest methodological lesson from this process was the importance of preliminary research.

My initial assumption was that there would be relatively little research on the use of AI across qualitative research, so I wanted to examine all qualitative research studies. The preliminary search showed that this assumption was wrong.

The search revealed the scale of the literature, helped identify the keywords I needed, and showed me which parameters I needed to set. I spent around three or four days on preliminary research, but it led to a scoping review that was much more focused and feasible.

This is something I see Masters students skip regularly. Students can propose systematic or scoping reviews that are far too broad simply because they have not investigated what the literature actually looks like.

For topics that I am not familiar with, I will do a quick search on an indexed database to establish whether the proposed study is feasible before proceeding further.

What Methodological Problems Can Appear During a Scoping Review?

Even after refining the review, I had concerns about the value and feasibility of the study.

So far, I had only examined abstracts and some full-text articles, and I was not seeing much information about how the AI systems were actually being used. It was possible that only a small number of papers would provide the information needed for the review.

I also encountered uncertainty around the definition of thematic analysis and how it was differentiated from content analysis and sentiment analysis. Some studies did not explicitly describe their approach as thematic analysis but still discussed coding and themes.

This creates a difficult methodological decision. I needed to consider how stringent the inclusion criteria should be while remembering the actual purpose of the review.

The review is intended to understand how AI can be used in thematic analysis, so I need to keep the research objectives in mind and approach these decisions with reflexivity. This is part of the nature of research: even when you have a strong understanding of a particular method, the literature may not always fit neatly into the categories you expected.

If you're applying this approach to your own research and want a second pair of eyes: https://www.drkenyanwong.com/start-here