systematic literature search

This study addresses the challenge of inadequate adherence to standardized reporting guidelines in scientific research, often caused by the time-consuming and error-prone nature of manual checklist completion, which undermines reproducibility. To overcome this, the authors propose an automated approach leveraging a locally deployed large language model (LLM), enhanced through instruction fine-tuning and a multi-stage prompting strategy, operating entirely on CPU-based inference. This method ensures data privacy and auditability while enabling cross-disciplinary checklist recommendation and item completion. For the first time, a local LLM is employed as a structured reasoning component for automated research reporting. Evaluated on a corpus of peer-reviewed manuscripts, the system achieves 90% accuracy in checklist recommendation and 88% accuracy in item completion, processing each manuscript in just 12.5 seconds without requiring GPU resources. The entire workflow is open-source and fully reproducible.

This work addresses the scarcity of legally compliant, reproducible, and high-quality full-text corpora in chemistry suitable for retrieval and text mining. The authors propose a reproducible construction pipeline that integrates legal compliance screening with technical validation, building upon the S2ORC dataset. By leveraging Unpaywall, OpenAlex, and Crossref to identify papers under open licenses, they assemble a corpus of over 580,000 chemistry articles. For each document, they generate paragraph-level embeddings using e5-large-v2, produce machine-generated summaries, and assign multi-label subfield annotations. The project releases complete code, metadata, and provenance artifacts to enable transparent reconstruction. Technical validation confirms high standards in text quality, embedding consistency, and metadata completeness, substantially enhancing the accessibility and reusability of chemical literature resources.

To address inconsistencies in ethical standards, protracted review processes, and variable assessment quality in Institutional Review Board (IRB) oversight, this paper introduces IRB-LLM—the first domain-specific large language model designed explicitly for IRBs. IRB-LLM integrates domain-adaptive fine-tuning, retrieval-augmented generation (RAG), and multi-task prompt engineering to establish a dynamic human-AI collaborative decision-making framework capable of semantically modeling ethical texts. It delivers three core functionalities: pre-review screening, consistency verification, and decision support—collectively enhancing both efficiency and standardization of ethical review. Experimental evaluation demonstrates that IRB-LLM reduces average processing time by 32% and improves inter-reviewer consistency in feedback by 27%. The model provides a reproducible, empirically validated paradigm for AI-augmented governance of research ethics.

This study addresses the time- and resource-intensive bottleneck of data extraction in scoping reviews by proposing an LLM-driven automation framework leveraging Claude 3.5 Sonnet and a structured review protocol. Methodologically, the protocol is directly embedded into system prompts, and an LLM-in-the-loop feedback mechanism is introduced to iteratively refine protocol design. Results show high accuracy (83.3%–100%) for simple, structured fields (e.g., author, year), but sharp declines for complex, subjective constructs (e.g., study aims, limitations)—with accuracy dropping to 9.6%–15.8% and overall F1 < 40%, primarily due to low recall (<25%). Protocol embedding improves inter-extraction consistency, and LLM feedback supports effective protocol refinement; however, current models exhibit insufficient reliability for semantic abstraction tasks. The study underscores the necessity of multi-dimensional performance evaluation and human-in-the-loop validation, offering both a methodological framework and pragmatic caution for deploying LLMs in evidence synthesis.

This work addresses the time-consuming and cognitively demanding process of drafting clinical trial eligibility criteria, which existing automated approaches often fail to support effectively due to reliance on structured inputs or limited controllability. The authors propose a guided generation framework grounded in semantic axes—such as demographics, laboratory values, and behavioral factors—that enables clinicians to steer large language models toward generating appropriate eligibility criteria without specifying exact entities. By incorporating an intermediate control mechanism and a reusable, multidimensional scoring system, the method strikes a balance between controllability and clinical utility. Experimental results demonstrate that the approach significantly outperforms unguided baselines across automatic metrics, standardized scoring rubrics, and clinician evaluations, thereby enhancing both the interpretability and practical value of AI-assisted clinical trial design.

This work addresses the limitations of existing reproducibility assessment methods, which rely on manual annotations and thus lack scalability and authentic supervision signals reflecting real-world reproduction challenges. The authors propose the first scalable evaluation framework that leverages GitHub user-submitted issues as natural supervision, enabling large-scale assessment of large language model (LLM) agents’ ability to identify paper-to-code reproducibility issues without human annotation. By integrating language understanding with code context analysis, the approach enables non-execution-based detection of reproducibility barriers. Experimental results demonstrate that the best-performing LLM agent identifies at least one semantically relevant reproducibility issue—aligned with those reported by humans—in approximately 90% of the evaluated papers, exhibiting strong performance in both failure detection and semantic localization.

This study addresses the challenge of irreproducibility in cybersecurity literature reviews caused by heterogeneous and dynamically evolving data sources. To overcome this, the authors propose a versioned, reproducible corpus system built upon DBLP as its metadata backbone, integrating open scholarly APIs and publisher-specific extractors to generate structured snapshots containing abstracts and BibTeX records. The system supports systematic review workflows through both command-line and web interfaces, transforming the corpus construction process into an executable, inspectable, and citable research artifact that enables auditable and repeatable measurements. A snapshot from May 2026 encompasses 9,925 papers from 11 top-tier conferences published between 2017 and 2026, achieving 99.86% abstract and 99.99% BibTeX coverage. Full-text keyword search exhibits sub-31-millisecond latency, and preprint analysis reveals that 29.2% of papers appeared on arXiv an average of five months before formal publication; incorporating author history filtering improves precision by 16.5× at 90% recall.

Peer review is experiencing exponential growth in volume, yet its quality remains highly variable, necessitating systematic, interpretable, and scalable evaluation tools. This work proposes PeeriScope—the first modular platform that integrates structured features, rubric-based large language model assessments, and supervised learning predictions to enable multidimensional, explainable quantification of review quality. PeeriScope supports diverse use cases including self-assessment by reviewers, editorial screening, and large-scale audit studies. Designed for real-world deployment and research extensibility, the platform offers an open API and a web interface. The project is open-sourced and accompanied by an online demo, aiming to foster continuous innovation and practical adoption of robust peer review evaluation methodologies.

The scale and complexity of modern scientific research render traditional peer review inadequate for effectively evaluating reproducibility. This work proposes a novel approach that frames reproducibility assessment as a structured reasoning task over scientific literature. It introduces an agent-based reasoning mechanism that leverages large language models to extract structured information, construct directed workflow graphs, and integrate multidimensional scoring for automated evaluation. The method enables consistent, cross-domain, and cross-model reconstruction of computational workflows, achieving accuracies of 61%, 60.71%, and 61.68% on the ReScience C, ReproBench, and GoldStandardDB benchmarks, respectively—substantially outperforming existing techniques.