An AI-supported scoping review still requires careful documentation, methodological judgement and transparent search processes. Losing database search histories can make a review impossible to replicate, but redoing the review can also reveal weaknesses in the original design and lead to a more rigorous final review. 

How did I lose a year of work on my scoping review?

The first review produced 46 included studies from 83 papers that reached full-text screening. I conducted manual and automated data extraction using Elicit AI, then combined and triangulated both extractions into one master extraction. Because the extraction columns were aligned with my research objectives, writing the results was largely a matter of turning those columns into narrative form.

The problem appeared only after I had drafted the full paper, which was about 11,000 words in the first draft. I realised that I had not properly saved my search histories on PubMed and Scopus.

The search strings and search results were gone. Although I had kept a research journal documenting amendments to my search strings, I had not saved the final versions that were actually run in the databases.

This meant I could not replicate the search. For a scoping review using a systematic approach, that was a fundamental problem. The only way to salvage the review was to rerun the searches and effectively redo the entire review.

I started the review in March 2024 and discovered the problem in February 2025. One full year of work was essentially gone.

What did I learn from redoing the scoping review?

Redoing the review gave me an opportunity to identify weaknesses that I could not have known about at the beginning.

First, I had originally narrowed the review from five years to three years to make it more manageable. After reading the papers, I found that recent papers were citing papers from five years earlier to justify their AI approaches. Returning to a five-year review therefore made the review better aligned with its purpose.

I also realised that my definition of thematic analysis had been too loose. Some papers referred to “themes” or “thematic” without explicitly describing their process as thematic analysis. Others conflated thematic analysis with content analysis or sentiment analysis. For greater accuracy and validity, I needed stricter exclusion criteria.

I also excluded papers that did not cite any material when describing their thematic analysis because I needed to identify which established thematic analysis processes researchers were pairing with AI.

The database selection also changed. PubMed and Scopus had initially seemed sufficient, but CINAHL could capture allied health and nursing research that I might otherwise have missed.

Finally, I removed two objectives. One concerned how researchers described their analysis methods and reported thematic analysis results, but this information was often constrained by journal word counts and reviewer feedback. Another concerned which qualitative methodology was used with AI, but most papers did not clearly declare their methodology, making the information difficult to extract.

How did I redo the scoping review?

The first review had shown that I was broadly on the right track, so much of the workflow remained the same. However, I refined the parameters and eligibility criteria, and the review became much more focused on how AI was used specifically within thematic analysis.

I reran the searches across PubMed, CINAHL and Scopus. For PubMed and CINAHL, I searched explicitly for the term “thematic analysis” in titles and abstracts and used tight proximity settings. This was intended to avoid papers that simply mentioned “themes” and had previously introduced content and sentiment analysis studies.

For the AI component, “artificial intelligence” as a MeSH term worked well. Additional AI-related terms, such as “neural networks”, had not retrieved unique papers in the previous review, so the MeSH term was sufficient.

On Scopus, I mirrored the search strategy but expanded the AI terms because MeSH is not used there. I also applied filters to focus on healthcare and excluded other scoping and systematic reviews because the review focused on empirical research.

I then followed the same screening workflow. I deduplicated the records in Rayyan, removing 237 duplicates, and uploaded 801 records to ASReview.

This time, I used a more rigorous stopping rule. Instead of stopping after 50 consecutive irrelevant records, I stopped after 100, which aligned better with similar reviews using ASReview.

Despite this stricter rule, I manually screened 364 records, compared with 387 previously. ASReview excluded 327 records and reached its stopping point faster, likely because the eligibility criteria were clearer and there were fewer grey-area papers. This saved approximately a day and a half of full-time screening.

After title and abstract screening, 110 papers passed and 106 full texts were retrieved. Four could not be obtained. Of the 106 full-text papers, 68 were excluded, mostly because AI was not used in the thematic analysis itself or was used only for peripheral tasks such as data cleaning or data collection.

I also screened papers from my previous repository using the revised criteria. Five were included from my own collection. Reference chaining identified 21 additional papers, of which one was included.

The final review contained 39 papers. Despite searching more databases, the review became smaller because the eligibility criteria were stricter and the focus was tighter.

How many papers should you include in a dissertation scoping review?

For a dissertation, 39 papers is a substantial amount of material to manage. This particular review focuses on research methods across healthcare, so a larger number of papers is expected.

For most dissertations, more papers mean more data to manage, synthesise and write about. You may not have enough time or word count to do this effectively.

I usually guide students towards a range of 12–25 papers by adjusting parameters such as the year range and eligibility criteria.

This involves trial and error. Testing and refining your search strategy early is therefore important because an initially broad search can easily produce hundreds of papers.

If you're applying this to your own work and want a second pair of eyes: https://www.drkenyanwong.com/start-here

Is redoing a scoping review a bad thing?

Redoing the review gave me the opportunity to make it stronger, more transparent and more rigorous, even though losing a year of work was frustrating.

This is also what I tell my Masters students: mistakes are part of learning to conduct research. If you register your review too early or set expectations around publication from the outset, you can create a situation where mistakes feel unacceptable.

That pressure can lead to endless tinkering with a search strategy or to a review that cannot ultimately be published because too many changes were made after the protocol.

In a Masters dissertation, you are allowed to make mistakes. You can learn from them with your supervisor, use feedback and results to revisit your decisions, and approach the work as a stronger researcher.