The Double-Edged Nature of the Rashomon Set for Trustworthy Machine Learning

2025-11-26

AI Summary

This paper investigates the dual role of the Rashomon set—the collection of near-optimal models—in trustworthy machine learning: it enhances robustness yet compromises privacy. We systematically analyze its impact on *reactive robustness* against adversarial attacks and distributional shifts, as well as on *privacy leakage* due to model disclosure, using both theoretical analysis and empirical evaluation on sparse decision trees and linear models. We formally characterize, for the first time, the quantitative trade-off between robustness and privacy induced by the Rashomon set, revealing that model diversity markedly improves stability under perturbations but substantially degrades differential privacy guarantees. Through interpretable modeling, adversarial testing, and distributionally robust evaluation, we empirically validate this “double-edged sword” effect. Finally, we propose principled guidelines for regulating the Rashomon set to advance trustworthy ML—balancing diversity, robustness, and privacy in model selection and deployment.

Technology Category

• Machine Learning: Adversarial Learning & Robustness
• Computer Vision: Adversarial Attacks & Robustness
• Natural Language Processing: Safety and Robustness

Application Category

• Security and Privacy: Security and privacy of machine learning and AI applications
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Real-world machine learning (ML) pipelines rarely produce a single model; instead, they produce a Rashomon set of many near-optimal ones. We show that this multiplicity reshapes key aspects of trustworthiness. At the individual-model level, sparse interpretable models tend to preserve privacy but are fragile to adversarial attacks. In contrast, the diversity within a large Rashomon set enables reactive robustness: even when an attack breaks one model, others often remain accurate. Rashomon sets are also stable under small distribution shifts. However, this same diversity increases information leakage, as disclosing more near-optimal models provides an attacker with progressively richer views of the training data. Through theoretical analysis and empirical studies of sparse decision trees and linear models, we characterize this robustness-privacy trade-off and highlight the dual role of Rashomon sets as both a resource and a risk for trustworthy ML.

Problem

• Characterizes the robustness-privacy trade-off in Rashomon sets
• Analyzes how model diversity affects adversarial attacks and data privacy
• Explores Rashomon sets as both a resource and risk for trustworthy ML

Innovation

• Analyzes Rashomon set's dual role in trustworthiness
• Demonstrates reactive robustness through model diversity
• Quantifies privacy-robustness trade-off via theoretical analysis