Score
Using R packages and workflows reproducibly, including implementing end-to-end analyses with packages like SelectiveIntegrative and intFRT, and running/diagnosing pipelines such as bkmr probit on simulated exposure-mixture datasets.
This study addresses the widespread lack of computational reproducibility in R supplementary code deposited on the Open Science Framework (OSF). A systematic audit of 296 published R code packages revealed that 98.8% incompletely declare dependencies. To address this, we propose the first automated reproducibility auditing framework tailored to the R ecosystem. It combines static source-code analysis—leveraging regular expressions and abstract syntax trees (ASTs)—to accurately infer dependencies, with Docker-based containerized execution and failure diagnostics (e.g., path errors, OS-specific inconsistencies, missing packages) to enable end-to-end environment reconstruction and validation. Experiments successfully executed 25.87% of scripts, identifying undeclared dependencies, hardcoded file paths, and cross-platform compatibility issues as the three primary barriers to reproducibility. The framework enables large-scale, low-cost, and scalable quantitative assessment of computational reproducibility in scholarly research, providing a practical toolchain to enhance transparency and verifiability.
Scientific computing artifacts—such as analysis scripts and software prototypes—frequently suffer from poor reproducibility due to environmental heterogeneity, dependency drift, and inadequate documentation, thereby undermining research credibility. To address this, we introduce the first cross-disciplinary, structured, and standardized benchmark dataset for computational experiments, encompassing workflows ranging from single-script executions to multi-language, complex pipelines. Our framework uniformly models metadata, standardizes dependency declarations (e.g., requirements.txt, Dockerfiles), encapsulates multi-language execution procedures, and prescribes a rigorous documentation protocol. The dataset comprises dozens of human-validated, fully reproducible experimental cases, enabling objective, comparable, and reproducible evaluation of reproducibility tools. This work fills a critical gap in the field by providing the first systematic, community-grounded benchmark for assessing computational reproducibility, thereby significantly enhancing the rigor, transparency, and comparability of reproducibility research.
This work proposes an AI agent–driven workflow to address the high costs of reproducing large-scale empirical studies, which often stem from discrepancies in computational environments, code, and documentation. The approach decouples scientific reasoning from computational execution: researchers supply standardized diagnostic templates, and the system automatically retrieves and orchestrates reproduction materials within a version-controlled environment. A structured knowledge layer captures failure patterns, enabling adaptive reproduction across heterogeneous studies while ensuring transparency and stability of the analytical pipeline. Evaluated on 92 instrumental variable studies, the method achieves an 87% end-to-end reproduction success rate; when data and code are available, it attains 100% success at both the paper and model levels.
Existing fairness toolkits predominantly focus on isolated interventions and neglect reproducibility and scalability. This paper introduces a modular, reproducibility-centered R-language machine learning workflow framework for fairness-aware modeling. Built upon a unified engine architecture, the framework integrates fairness methods as plug-and-play components across the entire pipeline—including data splitting, model training, and evaluation—ensuring transparency, auditability, and extensibility. It synergistically incorporates standards from Common Workflow Language (CWL) and Yet Another Workflow Language (YAWL), adopts KNIME’s visual workflow design principles, and leverages batchtools for lightweight parallel execution. The framework enables systematic integration, cross-method comparison, and standardized evaluation of fairness strategies. Moreover, it naturally extends to other trustworthy AI dimensions—including interpretability and robustness—making it suitable for high-compliance machine learning applications.
Addressing the poor reproducibility and high environmental heterogeneity of interdisciplinary computational experiments, this paper proposes SciRep—a framework that unifies management of code, data, programming languages, dependencies, and execution commands via containerized encapsulation, declarative experiment specifications (YAML/JSON), dependency snapshotting, and deterministic scheduling. It produces lightweight, portable “capsule packages.” SciRep introduces the first domain-agnostic reproducibility packaging paradigm, embodying “configuration-as-documentation” and “execution-as-verification,” and achieves end-to-end reproducible workflows across heterogeneous domains—including medicine, bioinformatics, and computer science—for the first time. It supports multi-language ecosystems (e.g., Python, R, Julia) on Linux/macOS. Empirical evaluation successfully reproduced 16 of 18 published experiments (89%), significantly surpassing the best prior tool’s 61% reproducibility rate; all successfully executed experiments reproduced original results with 100% fidelity.
This work addresses the widespread irreproducibility of academic Jupyter Notebooks caused by environment drift, missing dependencies, and implicit execution assumptions. The authors propose the first web-oriented, automated reproducibility engineering pipeline that systematically reconstructs and evaluates repository-level execution environments for notebooks hosted on GitHub. By leveraging dependency inference, auto-generated Docker containers, and isolated execution, the pipeline enables large-scale assessment of reproducibility. A novel four-category execution outcome framework is introduced to quantify reproduction fidelity. Evaluation on 443 real-world notebooks shows that containerization resolves 66.7% of dependency-related failures; however, only 46.3% achieve high output fidelity, demonstrating that while containerization is necessary, it is insufficient for bit-for-bit reproducibility. These findings underscore the critical need for systematic reproducibility evaluation in computational research.
This work addresses the challenge of conducting exact statistical inference within general parametric model families, where conventional methods often fall short. Building upon the Plausibility framework, the authors present the first systematic integration of this approach into the R programming environment by developing plausibility, an open-source R package implemented with object-oriented design. The package supports exact inference for a variety of penalized regression models, including glmnet, and features strong extensibility. Empirical evaluations on multiple real-world datasets demonstrate its computational efficiency and practical utility, substantially broadening the applicability of exact inference in modern regression analysis.
This study addresses a critical oversight in current practice wherein successful model fitting using the probit Bayesian Kernel Machine Regression (BKMR) implementation in the bkmr package is often mistakenly equated with Markov chain Monte Carlo (MCMC) convergence, without rigorous posterior diagnostic assessment. Through an extensive simulation study, the work systematically demonstrates that under standard settings, the probit BKMR model exhibits considerable fragility in achieving convergence: among 430 successfully fitted models, only 30 simultaneously satisfied stringent convergence criteria—namely, rank-normalized $\hat{R} \leq 1.01$, bulk effective sample size (bulk-ESS) $\geq 400$, and tail effective sample size (tail-ESS) $\geq 400$. Data were generated via bkmr::SimData() and models fitted using kmbayes(), with convergence evaluated through a multi-metric framework incorporating rank-normalized $\hat{R}$, bulk-ESS, and tail-ESS, underscoring the necessity of multidimensional diagnostics to ensure reliable inference.
This study addresses the challenge of integrating external control data in hybrid controlled trials, where covariate shift and outcome drift can compromise validity. The authors propose a comprehensive analytical framework that explicitly defines the target parameter and identification assumptions, and combines subject alignment, matching strategies, and selective borrowing to enhance statistical efficiency while preserving inferential validity. The approach integrates asymptotic inference with randomization-based testing and is fully reproducible through the SelectivelyIntegrative and intFRT software packages. In simulations emulating the CALGB 9633 lung cancer trial, the method maintains Type I error control while substantially improving estimation precision, demonstrating its practical utility and robustness for incorporating real-world evidence into clinical trial analysis.
This work proposes ReproScore, a novel framework that explicitly decouples reproducibility readiness (RRS) from reproducibility outcomes (ROS)—a distinction often conflated in existing tools that mistakenly treat static repository completeness as a proxy for successful execution. RRS comprises 26 fine-grained submetrics assessing code and documentation quality, while ROS is derived through sandboxed execution probes. The two dimensions are integrated into an adaptive composite score (RCS), with customizable metric weights via versioned YAML configurations contributed by the community. Empirical evaluation across 423 GitHub repositories spanning five failure modes demonstrates that environment-related metrics effectively discriminate failure types, yet reveals a near-zero correlation between RRS and actual execution success—exposing a critical “readiness–outcome gap” and thereby validating the necessity and efficacy of the proposed architectural separation.