🤖 AI Summary
The exponential growth of scientific literature underscores the urgent need for automated meta-analysis (AMA), yet existing approaches critically lack high-level synthesis capabilities—such as heterogeneity and bias assessment—and end-to-end automation. We systematically reviewed 978 publications (2006–2024) and conducted an in-depth analysis of 54 AMA studies, establishing the first comprehensive, lifecycle-spanning evaluation framework. Our analysis reveals that only 2% of studies achieve full workflow automation, with a pronounced application gap between medical (67%) and non-medical domains. To address these limitations, we propose a next-generation AMA paradigm integrating large language models (LLMs) with statistical rigor, synergizing NLP, explainable AI, and PRISMA-compliant methodology. Empirical results confirm that AMA substantially improves efficiency and reproducibility; however, robust, generalizable, fully automated meta-analysis remains unrealized. This work provides both a theoretical foundation and a concrete technical pathway toward overcoming this fundamental bottleneck.
📝 Abstract
Exponential growth in scientific literature has heightened the demand for efficient evidence-based synthesis, driving the rise of the field of Automated Meta-analysis (AMA) powered by natural language processing and machine learning. This PRISMA systematic review introduces a structured framework for assessing the current state of AMA, based on screening 978 papers from 2006 to 2024, and analyzing 54 studies across diverse domains. Findings reveal a predominant focus on automating data processing (57%), such as extraction and statistical modeling, while only 17% address advanced synthesis stages. Just one study (2%) explored preliminary full-process automation, highlighting a critical gap that limits AMA's capacity for comprehensive synthesis. Despite recent breakthroughs in large language models (LLMs) and advanced AI, their integration into statistical modeling and higher-order synthesis, such as heterogeneity assessment and bias evaluation, remains underdeveloped. This has constrained AMA's potential for fully autonomous meta-analysis. From our dataset spanning medical (67%) and non-medical (33%) applications, we found that AMA has exhibited distinct implementation patterns and varying degrees of effectiveness in actually improving efficiency, scalability, and reproducibility. While automation has enhanced specific meta-analytic tasks, achieving seamless, end-to-end automation remains an open challenge. As AI systems advance in reasoning and contextual understanding, addressing these gaps is now imperative. Future efforts must focus on bridging automation across all meta-analysis stages, refining interpretability, and ensuring methodological robustness to fully realize AMA's potential for scalable, domain-agnostic synthesis.