Why are there many equally good models? An Anatomy of the Rashomon Effect

2026-01-11

arXiv.org

AI Summary

This study investigates the fundamental causes underlying the Rashomon effect in machine learning—the phenomenon where multiple models with similar performance yet distinct structures coexist. For the first time, it systematically categorizes these causes into three types: statistical (due to finite samples and noise), structural (arising from non-convexity and non-identifiability of the target), and procedural (stemming from limitations of optimization algorithms and constraints of the model class). The work clarifies their essential differences: statistical multiplicity diminishes with more data, whereas structural multiplicity requires additional assumptions or new data to resolve. By integrating perspectives from machine learning, statistics, and optimization theory, the paper establishes a unified analytical framework that elucidates the origins of this multiplicity, offering theoretical foundations and practical insights for model interpretability, fairness, and decision-making under uncertainty.

Technology Category

• Machine Learning: Causal Learning
• Reasoning under Uncertainty: Causality
• Knowledge Representation and Reasoning: Computational Complexity of Reasoning

Application Category

• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems

Abstract

The Rashomon effect -- the existence of multiple, distinct models that achieve nearly equivalent predictive performance -- has emerged as a fundamental phenomenon in modern machine learning and statistics. In this paper, we explore the causes underlying the Rashomon effect, organizing them into three categories: statistical sources arising from finite samples and noise in the data-generating process; structural sources arising from non-convexity of optimization objectives and unobserved variables that create fundamental non-identifiability; and procedural sources arising from limitations of optimization algorithms and deliberate restrictions to suboptimal model classes. We synthesize insights from machine learning, statistics, and optimization literature to provide a unified framework for understanding why the multiplicity of good models arises. A key distinction emerges: statistical multiplicity diminishes with more data, structural multiplicity persists asymptotically and cannot be resolved without different data or additional assumptions, and procedural multiplicity reflects choices made by practitioners. Beyond characterizing causes, we discuss both the challenges and opportunities presented by the Rashomon effect, including implications for inference, interpretability, fairness, and decision-making under uncertainty.

Problem

• Rashomon effect
• model multiplicity
• non-identifiability
• predictive equivalence
• machine learning

Innovation

• Rashomon effect
• model multiplicity
• non-identifiability
• statistical learning
• optimization landscape