Directional Influence Function: Estimating Training Data Influence in Constrained Learning

2026-07-25

AI Summary

Traditional influence functions fail in constrained learning settings because they neglect how data perturbations affect the feasible region, leading to biased estimates or infeasible solutions. This work proposes Directional Influence Functions (DIF), which explicitly incorporate constraints into the influence analysis framework for the first time. By modeling the optimality conditions of constrained optimization as a variational inequality and integrating sensitivity analysis with leave-one-out approximations, DIF accurately captures the effect of training sample perturbations on model parameters. Experiments on constrained linear regression and CNNs with fairness constraints demonstrate that DIF precisely replicates retraining results, significantly outperforming classical influence functions and their penalty-based variants, and exhibits strong alignment with actual retraining outcomes in predicting changes in test loss.

Abstract

As constrained learning becomes increasingly common, models are trained under explicit feasibility requirements to enforce fairness, safety, robustness, regulariza- tion, and physics or logic constraints. Understanding how training samples in- fluence the model solution (e.g., learned parameters) is crucial for interpretability and robustness. The classical influence function (IF) estimates sample contribu- tions via local sensitivity analysis, measuring how the solution changes when a specific training sample is perturbed or removed. However, IF becomes unreli- able in constrained settings: data perturbations can reshape both the objective and the feasible region, leading to estimates that violate feasibility. In response, we propose the Directional Influence Function (DIF), a novel estimator that explicitly incorporates these constraints into influence estimation. DIF formulates the opti- mality conditions of constrained learning as a variational inequality (VI) and ana- lyzes how perturbing training data affects this VI. We validate DIF on constrained linear regression and demonstrate that it recovers leave-one-out retraining results, whereas IF and penalty-based IF exhibit significant bias. We further apply DIF to fairness-constrained CNNs, where DIF accurately predicts test loss changes under data removal and aligns closely with actual retraining. Our results establish DIF as an efficient and reliable tool for data attribution in constrained learning.

Problem

• constrained learning
• influence function
• data attribution
• feasibility constraints
• model interpretability

Innovation

• Directional Influence Function
• Constrained Learning
• Variational Inequality
• Data Attribution
• Influence Estimation