SHAP-Guided Regularization in Machine Learning Models

2025-07-31

AI Summary

This work addresses the longstanding challenge of jointly optimizing model interpretability and predictive performance in machine learning. We propose a SHAP-driven regularized training framework, whose core innovation is the first direct incorporation of TreeSHAP attribution values into the loss function via a novel joint regularization term based on the entropy of the attribution distribution. This term simultaneously enforces sparsity, concentration, and cross-sample stability of feature importances. Unlike post-hoc methods, our approach is end-to-end trainable, applicable to mainstream tree-based models (e.g., XGBoost, LightGBM), and supports both regression and classification tasks. Extensive experiments across multiple benchmark datasets demonstrate that the proposed method improves model generalization, yields more robust and interpretable SHAP attributions, and maintains or exceeds baseline accuracy—without sacrificing predictive performance.

Technology Category

• Machine Learning: Transparent, Interpretable, Explainable ML
• Humans and AI: Explainable AI (XAI) for Human Understanding
• Natural Language Processing: Interpretability, Analysis, and Evaluation of NLP Models

Application Category

• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Feature attribution methods such as SHapley Additive exPlanations (SHAP) have become instrumental in understanding machine learning models, but their role in guiding model optimization remains underexplored. In this paper, we propose a SHAP-guided regularization framework that incorporates feature importance constraints into model training to enhance both predictive performance and interpretability. Our approach applies entropy-based penalties to encourage sparse, concentrated feature attributions while promoting stability across samples. The framework is applicable to both regression and classification tasks. Our first exploration started with investigating a tree-based model regularization using TreeSHAP. Through extensive experiments on benchmark regression and classification datasets, we demonstrate that our method improves generalization performance while ensuring robust and interpretable feature attributions. The proposed technique offers a novel, explainability-driven regularization approach, making machine learning models both more accurate and more reliable.

Problem

• Incorporates SHAP-guided regularization to enhance predictive performance and interpretability
• Applies entropy-based penalties for sparse, stable feature attributions
• Improves generalization performance with robust, explainable models

Innovation

• SHAP-guided regularization for model optimization
• Entropy-based penalties for sparse feature attributions
• TreeSHAP for tree-based model regularization