Position: Let's Strengthen Verifiability If We Can't Enforce Reproducibility

2026-09-25

AI Summary

This study addresses the impediments to progress in machine learning caused by poor reproducibility of results and unavailable code by conducting a quantitative analysis of the reproducibility dilemma. Methodologically, it departs from the traditional paradigm that insists on enforcing exact replication, instead advocating for enhanced inspectability and transparency of research findings through empirical analysis and the establishment of community standards. The core contribution lies in proposing a concrete framework that substitutes mandatory reproduction with strengthened result verifiability. Furthermore, this work releases open-source supporting materials and actionable recommendations for improvement. By offering a pragmatic pathway to overcome persistent reproducibility challenges, this project effectively promotes greater transparency and standardization within artificial intelligence research.

Abstract

In the field of Machine Learning, many papers contain empirical results supporting claimed statements or illustrating the performance of a proposed method. However, most practitioners know that (1) results are generally hard to reproduce, and increasingly so, (2) code is not often available to do so, and (3) it hinders the development of research. In this position paper, we analyze and quantify these issues, and make concrete proposals to improve result checkability, if not reproducibility. Code and supporting materials are available at https://github.com/giddyyupp/position-enforce-verifiability.

Problem

• Machine Learning
• Reproducibility
• Verifiability
• Empirical Results
• Code Availability

Innovation

• Verifiability
• Reproducibility
• Machine Learning
• Position Paper
• Empirical Results