🤖 AI Summary
This study addresses the challenge of inadequate adherence to standardized reporting guidelines in scientific research, often caused by the time-consuming and error-prone nature of manual checklist completion, which undermines reproducibility. To overcome this, the authors propose an automated approach leveraging a locally deployed large language model (LLM), enhanced through instruction fine-tuning and a multi-stage prompting strategy, operating entirely on CPU-based inference. This method ensures data privacy and auditability while enabling cross-disciplinary checklist recommendation and item completion. For the first time, a local LLM is employed as a structured reasoning component for automated research reporting. Evaluated on a corpus of peer-reviewed manuscripts, the system achieves 90% accuracy in checklist recommendation and 88% accuracy in item completion, processing each manuscript in just 12.5 seconds without requiring GPU resources. The entire workflow is open-source and fully reproducible.
📝 Abstract
Transparent and standardized reporting is essential for reproducible scientific research, yet adherence to reporting guidelines remains inconsistent because of the manual effort required to select and complete checklists. We present CheckSupport, an open-source, locally deployable system that uses large language models to automate the recommendation of reporting checklists and the evidence-grounded completion of checklists for scientific manuscripts. CheckSupport employs a staged prompting strategy that decomposes reporting workflows into constrained inference tasks, prioritizing faithful extraction over generative text synthesis. All inference is performed locally using instruction-tuned models, preserving data privacy and enabling reproducible, auditable workflows. Evaluated on a corpus of peer-reviewed manuscripts, CheckSupport achieved 90% overall accuracy for checklist recommendations and 88% overall accuracy for item-level completion while operating on CPU-only hardware. On average, the wall-clock time per manuscript was 12.5 seconds, including the checklist recommendation and full checklist completion. These results demonstrate that large language models, when applied as structured inference components, can reduce reporting burden and support more transparent and reproducible scientific reporting across disciplines.