Learning Samples Importance: Parameterizing Dual Variables in Everywhere Learning

2026-09-28

AI Summary

This study addresses the limitation of the learning-to-learn-everywhere framework, where dual variables fail to generalize to unseen samples. To overcome this, we propose, for the first time, parameterizing dual variables as functions of the input samples. Methodologically, parametric function approximation techniques are employed to model statistical multipliers, and recovery errors are analyzed within the framework of constrained optimization theory. Experimental results demonstrate that the proposed method efficiently solves constrained problems. Its core contribution lies in endowing dual variables with generalization capabilities beyond the training set, enabling them to provide meaningful sample-level sensitivity characterizations for unseen instances. This effectively overcomes the bottleneck associated with traditional independent multipliers.

Abstract

Everywhere learning provides a principled framework for training AI models under constraints that must hold throughout the data distribution. In the dual domain, these pointwise constraints give rise to functional dual variables. In this work, we propose to learn these dual variables, motivated by the fact that their values encode useful information about the underlying constrained problem. By representing the dual variable as a parametric function of each sample, we enable the learned multiplier to be evaluated on new, unseen samples. This contrasts with standard empirical dual formulations, which assign an independent multiplier to each training sample. We characterize the error in the recovered primal solution induced by restricting the dual variable to a parametric function class and show that it is controlled by how well this class approximates the optimal statistical multiplier. Moreover, we show that the learned parametric multiplier retains the sensitivity interpretation of the optimal statistical multiplier, yielding approximate sensitivity guarantees that extend beyond the samples used for training. We empirically validate our theory across a variety of everywhere learning tasks, showing that the resulting constrained problems can be solved efficiently and that the learned dual variables provide meaningful representations of sample-level sensitivity.

Problem

• Everywhere learning
• Dual variables
• Parameterization
• Constrained optimization
• Sample importance

Innovation

• Everywhere learning
• Dual variables parameterization
• Functional dual variables
• Sensitivity interpretation
• Constrained optimization