Mixtures of Transparent Local Models

2026-01-15

AI Summary

This work proposes a hybrid framework that balances model performance and interpretability by composing multiple transparent local models. Specifically, simple and interpretable label functions are learned over distinct regions of the input space, and both the local models and their corresponding regions of applicability are jointly optimized via a novel multi-predictor, multi-region loss function. The approach uniquely integrates locally transparent modeling with PAC-Bayesian theory, establishing rigorous generalization risk bounds for both binary classification and regression tasks. Experimental results demonstrate the method’s validity on synthetic data, outperform existing interpretable methods on real-world datasets, and achieve performance comparable to certain black-box models.

Technology Category

Machine Learning: Transparent, Interpretable, Explainable MLComputer Vision: Interpretability, Explainability, and TransparencyNatural Language Processing: Interpretability, Analysis, and Evaluation of NLP Models

Application Category

User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalizationSearch and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for rankingSemantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

The predominance of machine learning models in many spheres of human activity has led to a growing demand for their transparency. The transparency of models makes it possible to discern some factors, such as security or non-discrimination. In this paper, we propose a mixture of transparent local models as an alternative solution for designing interpretable (or transparent) models. Our approach is designed for the situations where a simple and transparent function is suitable for modeling the label of instances in some localities/regions of the input space, but may change abruptly as we move from one locality to another. Consequently, the proposed algorithm is to learn both the transparent labeling function and the locality of the input space where the labeling function achieves a small risk in its assigned locality. By using a new multi-predictor (and multi-locality) loss function, we established rigorous PAC-Bayesian risk bounds for the case of binary linear classification problem and that of linear regression. In both cases, synthetic data sets were used to illustrate how the learning algorithms work. The results obtained from real data sets highlight the competitiveness of our approach compared to other existing methods as well as certain opaque models. Keywords: PAC-Bayes, risk bounds, local models, transparent models, mixtures of local transparent models.

Problem

• transparent models
• local models
• mixtures of local transparent models
• interpretability
• model transparency

Innovation

• PAC-Bayes
• risk bounds
• local models
• transparent models
• mixtures of local transparent models