Proactive Routing to Interpretable Surrogates with Distribution-Free Safety Guarantees

2026-03-15

AI Summary

This work addresses the critical challenge of safely replacing black-box models with low-cost, interpretable proxy models during deployment while ensuring controlled performance degradation. We propose an active routing mechanism that employs a lightweight gating model to decide at inference time whether to invoke the proxy, coupled with Clopper–Pearson conformal calibration to rigorously bound the rate of performance degradation violations without any distributional assumptions. To our knowledge, this is the first approach to provide distribution-free safety guarantees for model routing, establishing theoretical feasibility conditions and an AUC-based threshold criterion. We further show that probabilistic calibration primarily affects routing efficiency rather than validity. Experiments across 35 OpenML datasets demonstrate that our method substantially outperforms regression-based conformal and naive baselines, achieving significantly higher proxy usage coverage while strictly controlling violation rates.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Planning, Routing, and Scheduling: Model-Based Reasoning
• Natural Language Processing: Safety and Robustness

Application Category

• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Security and Privacy: Large-scale security measurements
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

Model routing determines whether to use an accurate black-box model or a simpler surrogate that approximates it at lower cost or greater interpretability. In deployment settings, practitioners often wish to restrict surrogate use to inputs where its degradation relative to a reference model is controlled. We study proactive (input-based) routing, in which a lightweight gate selects the model before either runs, enabling distribution-free control of the fraction of routed inputs whose degradation exceeds a tolerance τ. The gate is trained to distinguish safe from unsafe inputs, and a routing threshold is chosen via Clopper-Pearson conformal calibration on a held-out set, guaranteeing that the routed-set violation rate is at most α with probability 1-δ. We derive a feasibility condition linking safe routing to the base safe rate π and risk budget α, along with sufficient AUC thresholds ensuring that feasible routing exists. Across 35 OpenML datasets and multiple black-box model families, gate-based conformal routing maintains controlled violation while achieving substantially higher coverage than regression conformal and naive baselines. We further show that probabilistic calibration primarily affects routing efficiency rather than distribution-free validity.

Problem

• model routing
• interpretable surrogates
• distribution-free guarantees
• safety constraints
• model degradation

Innovation

• proactive routing
• conformal calibration
• distribution-free guarantee
• interpretable surrogate
• model degradation control