Systematic Literature Reviews With Two Multi-Agentic Systems And Human-In-The-Loop

2026-07-23

AI Summary

Traditional systematic literature reviews suffer from low efficiency and susceptibility to selection bias during clinical trial screening and data extraction. This work proposes two task-oriented multi-agent systems (MAS)—one for automated trial screening and another for structured information extraction—both integrating human-in-the-loop mechanisms to support clinical decision-making. The systems innovatively employ heterogeneous large language model agents, multi-round cross-review, standardized workflows, retrieval-augmented context control, and iterative error correction, substantially enhancing accuracy and scalability. In a real-world network meta-analysis replication, the approach not only fully reproduced all trials included in the original study but also identified additional eligible trials missed by manual screening, thereby updating the clinical conclusions.

Abstract

Systematic literature review of clinical trials drives regulatory decision-making, but conventional screening and extraction are time-consuming, labor-intensive, and vulnerable to study selection bias. We propose two fit-to-purpose multi-agentic systems (MAS) for systematic literature review, with human-in-the-loop. The screening MAS uses multiple LLM agents with heterogeneous personas and multiround cross-review, and uniformly improves accuracy over a single-LLM baseline. The extraction MAS combines standardization, an iterative correction loop, and retrieval-based context control to ensure accuracy and scalability. Both MAS are specifically designed to support Human-In-The-Loop which is essential for clinical decisions. The novelty of the proposed approach lies in the system architecture rather than in any single foundation tools: the system can naturally benefit from future improvements in the underlying tools, for instance, stronger LLM agents, retrieval engines, image recognition methods, etc. As a real-world application, a published network meta-analysis is reproduced by the MAS. The result recovers all trials from the original study and identifies additional eligible trials missed by manual review, leading to updated clinical conclusions.

Problem

• systematic literature review
• clinical trials
• study selection bias
• screening
• data extraction

Innovation

• multi-agentic systems
• human-in-the-loop
• systematic literature review
• large language models
• clinical trial extraction