Syst Rev ; 8 1 :Nov The goal of our project was to conduct a case study to explore a screening approach that temporarily replaces a human screener with a semi-automated screening tool.
A published comparative effectiveness review served as the reference standard. Five teams of professional systematic reviewers screened the same abstracts in parallel.
Each team trained DistillerAI with randomly selected abstracts that the team screened dually. For all remaining abstracts, DistillerAI replaced one human screener and provided predictions about the relevance dysbiosis opposite records.
A single reviewer also screened all remaining dysbiosis opposite. A second human screener resolved conflicts between the single reviewer and DistillerAI.
We compared the decisions of the machine-assisted approach, single-reviewer screening, and screening with DistillerAI alone against the reference standard. Machine-assisted screening and single-reviewer screening had similar areas under the curve 0. The interrater agreement between human screeners and DistillerAI with a prevalence-adjusted kappa dysbiosis opposite 0.
Rapid reviews, which do not require detecting the totality of the relevant evidence, may find semi-automation tools to have greater utility than traditional systematic reviews.