Score
Developing reproducible statistical analyses and software in R, including implementing multiple testing procedures, sensitivity methods, and practitioner-ready scripts for applied research.
Scientific software testing faces unique challenges—including difficult test case design, ambiguous oracle determination, absence of quality assessment standards, and poor applicability of industrial testing tools. Method: We conducted the first large-scale empirical study, combining structured surveys with qualitative analysis and statistical testing across 217 scientific software developers to examine variations in testing practices, tool adoption, and demographic factors. Contribution/Results: We identify three core bottlenecks: test design, result validation, and quality measurement; further reveal widespread lack of awareness of and access to domain-specific testing tools. This work establishes the first empirical evidence of paradigmatic divergence between scientific and conventional software testing, advocating for lightweight, extensible, and computationally aware testing frameworks tailored to scientific computing. Our findings provide foundational evidence and strategic direction for advancing domain-specific testing methodology.
This study addresses the widespread lack of computational reproducibility in R supplementary code deposited on the Open Science Framework (OSF). A systematic audit of 296 published R code packages revealed that 98.8% incompletely declare dependencies. To address this, we propose the first automated reproducibility auditing framework tailored to the R ecosystem. It combines static source-code analysis—leveraging regular expressions and abstract syntax trees (ASTs)—to accurately infer dependencies, with Docker-based containerized execution and failure diagnostics (e.g., path errors, OS-specific inconsistencies, missing packages) to enable end-to-end environment reconstruction and validation. Experiments successfully executed 25.87% of scripts, identifying undeclared dependencies, hardcoded file paths, and cross-platform compatibility issues as the three primary barriers to reproducibility. The framework enables large-scale, low-cost, and scalable quantitative assessment of computational reproducibility in scholarly research, providing a practical toolchain to enhance transparency and verifiability.
This work addresses the inefficiencies in oncology clinical trial statistical workflows—often fragmented, leading to redundant efforts, poor collaboration, and inconsistent analyses—by developing grstat, an open-source R package that integrates standardized analytical tools within a governance framework featuring requirement traceability, peer review, automated testing, and phased validation. By unifying technical implementation with a structured, reproducible process, grstat establishes a shared, auditable, and maintainable analytical toolkit. Empirical application demonstrates that this approach substantially enhances analytical efficiency, consistency, and long-term maintainability, offering academic biostatistics teams a scalable and transferable collaborative paradigm.
Scientific computing artifacts—such as analysis scripts and software prototypes—frequently suffer from poor reproducibility due to environmental heterogeneity, dependency drift, and inadequate documentation, thereby undermining research credibility. To address this, we introduce the first cross-disciplinary, structured, and standardized benchmark dataset for computational experiments, encompassing workflows ranging from single-script executions to multi-language, complex pipelines. Our framework uniformly models metadata, standardizes dependency declarations (e.g., requirements.txt, Dockerfiles), encapsulates multi-language execution procedures, and prescribes a rigorous documentation protocol. The dataset comprises dozens of human-validated, fully reproducible experimental cases, enabling objective, comparable, and reproducible evaluation of reproducibility tools. This work fills a critical gap in the field by providing the first systematic, community-grounded benchmark for assessing computational reproducibility, thereby significantly enhancing the rigor, transparency, and comparability of reproducibility research.
Scientific software frequently suffers from poor robustness, low maintainability, and weak sustainability. To address these challenges, this work systematically integrates software engineering best practices with domain-specific research requirements, proposing a set of ten high-quality principles for building scientific software across its entire lifecycle. The principles cover critical phases—including project planning, readable coding, version control, automated testing, modular design, reproducibility assurance, performance optimization, and long-term maintenance—and are supported by technical enablers such as automated documentation generation, continuous integration, and performance profiling. Designed to be both broadly applicable and practically actionable, the framework has been empirically validated across multiple scientific domains. Results demonstrate significant improvements in software reliability, reusability, and collaborative efficiency within research communities, thereby enhancing the academic impact of scientific tools and advancing open science and reproducible research ecosystems.
Doctoral students in life sciences commonly lack formal software engineering training, hindering the development of robust, reproducible, and collaborative research software. Method: This study proposes ten pedagogical principles for research software development, establishing the first systematic framework centered on “research software pedagogy”—distinct from generic programming instruction. It integrates software engineering best practices (e.g., Git-based version control, CI/CD pipelines, unit testing, RESTful API design), learning science principles, and authentic research workflows, emphasizing the seamless embedding of automation, documentation, testing, and collaborative practices throughout the research lifecycle. Contribution/Results: The framework delivers a generalizable, plug-and-play pedagogical paradigm. Deployed across multiple Chinese universities’ life sciences PhD programs, it has demonstrably improved software deliverable quality, code reusability, and cross-team collaboration efficiency—bridging critical gaps between computational literacy and rigorous, team-based scientific software practice.
To address the lack of early, machine-readable descriptions of scientific data analysis workflows—hindering FAIR (Findable, Accessible, Interoperable, Reusable) compliance—this paper introduces dtreg, the first structured registration framework for statistical and machine learning pipelines targeting the pre-publication stage and supporting both Python and R. Its core contributions are: (1) a novel pre-analysis metadata registration mechanism; (2) a persistent, globally identifiable schema system covering mainstream statistical tests (e.g., t-tests) and ML methods; and (3) lightweight, automated RDF/Linked Data serialization to Turtle and JSON-LD. Leveraging object-oriented modeling, dynamic schema population, and export capabilities, dtreg enables end-to-end machine-readable workflow documentation. As an open-source infrastructure, it significantly enhances the findability, interoperability, and reusability of analytical methods in computational research.
This work proposes an AI agent–driven workflow to address the high costs of reproducing large-scale empirical studies, which often stem from discrepancies in computational environments, code, and documentation. The approach decouples scientific reasoning from computational execution: researchers supply standardized diagnostic templates, and the system automatically retrieves and orchestrates reproduction materials within a version-controlled environment. A structured knowledge layer captures failure patterns, enabling adaptive reproduction across heterogeneous studies while ensuring transparency and stability of the analytical pipeline. Evaluated on 92 instrumental variable studies, the method achieves an 87% end-to-end reproduction success rate; when data and code are available, it attains 100% success at both the paper and model levels.
This study addresses the critical yet underexplored issue of how calibration and dichotomization thresholds in Qualitative Comparative Analysis (QCA) substantially influence analytical outcomes, while existing approaches lack systematic and efficient tools for sensitivity analysis. To bridge this gap, we introduce TSQCA, an R package that explicitly treats thresholds as analytical variables. TSQCA implements four sweep functions—otSweep, ctSweepS, ctSweepM, and dtSweep—to automate the exploration of multidimensional threshold combinations and their effects on QCA results. Built upon the CRAN QCA package for truth table construction and Boolean minimization, TSQCA employs an S3 object system to standardize output formats and supports automated generation of reproducible Markdown reports and visualizations. This framework significantly enhances the robustness, transparency, and reproducibility of QCA research.
This study addresses the lack of user-friendly, regulatory-compliant software tools for covariate adjustment in clinical trials aligned with the latest FDA guidance. To bridge this gap, the authors developed the R packages RobinCar and RobinCar2, which unify both conventional and state-of-the-art methods—including ANCOVA, G-computation, and PROCOVA™—within a single analytical framework. These tools support covariate-adjusted analyses for continuous, discrete, and time-to-event outcomes, implementing regulatory-endorsed approaches such as generalized linear models, machine learning algorithms, augmented covariate adjustment, covariate-adjusted log-rank tests, and marginal hazard ratio estimation. Validation on the ACTG 175 dataset demonstrates their computational efficiency and ease of use, substantially enhancing accessibility to analyses that meet current regulatory standards.
This study addresses longstanding challenges in agricultural experimentation, where traditional statistical analyses often suffer from a disconnect between experimental design and computational implementation, leading to subjective model specification, mischaracterized error structures, and biased interpretation of interaction effects. To resolve these issues, the authors propose the first declarative Python framework that deeply embeds experimental design semantics into the analytical workflow. The framework automatically translates complex designs—such as randomized complete blocks, split-plots, and multi-environment trials—into valid linear or mixed-effects models, accurately identifies error strata, performs hypothesis tests and mean comparisons, and unifies ANOVA, mixed-model inference, and stability analysis. By rigorously enforcing correct interpretation of hierarchical and interaction effects, the approach maintains consistency with classical methodologies while substantially enhancing inferential accuracy, reproducibility, and reliability.