noise injection

Machine learning training is significantly more time-consuming than inference, and the design of input or parameter perturbations has long relied on empirical trial-and-error. Method: This paper models training dynamics as a first-passage process and introduces a statistical mechanics framework to analyze model responses to input/parameter perturbations. It proposes, for the first time, a single-frequency perturbation response theory grounded in the quasi-stationary assumption, and rigorously proves its generalizability to multi-frequency perturbation regimes—enabling rational optimization of perturbation protocols. Contribution/Results: Evaluated on ResNet-18 trained for CIFAR-10 classification, the method precisely identifies the optimal perturbation type and frequency, reducing training iterations by 23% and improving test accuracy by 1.4 percentage points, thereby substantially enhancing both training efficiency and generalization performance.

This study addresses the robustness of global post-hoc interpretability methods—such as Partial Dependence Plots (PDP) and Accumulated Local Effects (ALE)—for black-box predictive models on tabular data under data and model perturbations. Motivated by reliability deficits arising from their fragility in model debugging and scientific discovery, we propose the first theoretical bounds quantifying explanation deviation under worst-case and best-case perturbations. Our approach integrates rigorous theoretical analysis with extensive empirical evaluation across multiple real-world and synthetic datasets, systematically assessing explanation sensitivity to controlled perturbations. Results demonstrate that even minor perturbations can substantially distort global feature effect estimates—up to complete reversal of effect direction in adversarial cases. This work establishes the first formal robustness framework for global interpretability, filling a critical gap in the theoretical foundations of trustworthy XAI. It provides the first quantifiable diagnostic benchmark for evaluating the stability and fidelity of global explanations.

Offline optimization of expensive black-box functions in materials engineering suffers from poor robustness due to the high sensitivity of surrogate models to parameter perturbations. Method: We propose, for the first time, an optimizable surrogate sensitivity metric and design a sensitivity-aware regularization method orthogonal to existing frameworks. This approach integrates gradient-based sensitivity analysis with deep-learning-based surrogate modeling and is compatible with mainstream paradigms such as offline Bayesian optimization. Contribution/Results: Evaluated on multiple materials design benchmarks, our method significantly improves optimization success rate (average gain of +23.6%) and solution quality (objective value improvement up to 17.4%). Empirical results demonstrate that explicit sensitivity control delivers critical performance gains for offline optimization of expensive black-box functions in materials engineering.

Differential privacy (DP) gradient training suffers from excessive noise injection and suboptimal privacy–utility trade-offs due to reliance on global sensitivity, which is overly conservative for modern deep models. Method: This paper proposes the first scalable and verifiable framework for computing upper bounds on both local and smooth sensitivity—novelly integrating convex relaxation with interval-bound propagation to enable precise, efficient estimation of smooth sensitivity during gradient computation in contemporary deep neural networks. Contribution/Results: Our approach overcomes longstanding theoretical and computational barriers in rigorously bounding sensitivity. Experiments across financial risk assessment, medical image classification, and multi-task NLP demonstrate that our method reduces required noise magnitude by an order of magnitude, yielding substantial improvements in prediction accuracy and practical utility under identical privacy budgets (e.g., ε = 2, δ = 10⁻⁵). The framework provides stronger theoretical guarantees for private inference while ensuring engineering feasibility and scalability.

To address the prevalent reward collapse problem in diffusion model fine-tuning, this paper proposes an entropy-regularized stochastic control framework and— for the first time—rigorously extends it to general *f*-divergence regularization. Methodologically, we formulate a continuous-time stochastic control model, integrating Itô calculus with variational inference to derive a computationally tractable and provably convergent optimal control policy. Theoretically, we establish that the proposed regularization effectively mitigates reward collapse; empirically, it significantly improves both sample quality and diversity. Key contributions include: (1) the first rigorous stochastic control analysis framework specifically designed for diffusion model fine-tuning; (2) a unified generalization of entropy regularization to arbitrary *f*-divergences, substantially enhancing methodological generality and robustness; and (3) a practical fine-tuning paradigm implementable under multiple divergence metrics.

This study investigates the relationship between the robustness of neural networks under random input perturbations and their prediction accuracy, measured by mean squared error (MSE). To address this, the work proposes an efficient, computable black-box robustness metric that, without requiring access to internal model architecture, provides a high-probability upper bound on the network’s MSE over an entire dataset under a given perturbation. The method innovatively introduces robustness curves, enabling systematic comparison and analysis of robustness across different datasets. Experimental evaluations on multiple real-world datasets demonstrate that the proposed approach accurately quantifies and effectively captures a model’s sensitivity to input noise, offering a practical tool for assessing robustness in diverse settings.

This work addresses the excessive noise introduced by existing Pufferfish privacy mechanisms due to overly stringent constraints, which severely degrades data utility. The authors propose a relaxed noise calibration approach that integrates the 1-Wasserstein (Kantorovich) mechanism, prior belief modeling, and ℓ₁-sensitivity theory to construct a general and practical mechanism generation algorithm. Theoretically, they prove that for any privacy budget and prior, there exists a strictly superior noise reduction scheme, yielding substantially improved utility—especially under low privacy budgets—and that the worst-case 1-Wasserstein mechanism is equivalent to the ℓ₁-sensitivity method. Empirical evaluations on three real-world datasets demonstrate utility gains of 47%–87%, confirming the method’s broad applicability and superiority.

This work investigates the impact of parameter noise injection in stochastic gradient descent on optimization and generalization, emphasizing the need for efficient per-sample perturbations and sophisticated noise schemes. By leveraging distributional identities of linear layers, the authors propose a method that enables per-sample noise injection within mini-batches without disrupting batched computation. They systematically compare isotropic and diagonal Gaussian noise variants, demonstrating that on CIFAR-100, a lightweight single-sample isotropic Gaussian perturbation recovers most of the optimization and generalization benefits achieved by more complex multi-sample strategies. These findings suggest that simplified noise injection designs can be sufficiently effective, offering a practical alternative to computationally heavier approaches while maintaining performance gains.

This work addresses the limitations of traditional generalization analyses, which rely on the often unverifiable assumption of independent and identically distributed (i.i.d.) data and thus struggle to accurately characterize model performance on unseen data. The paper proposes a deterministic generalization analysis framework that dispenses with any prior probabilistic assumptions. By examining the sensitivity of optimization solutions to data perturbations, it decomposes the generalization error into geometric and probabilistic components, achieving their first-ever decoupling. The framework expresses generalization bounds via a variational principle, leveraging deterministic perturbation analysis and optimization sensitivity theory to capture the discrepancy between in-sample and out-of-sample performance. Error terms are evaluated through posterior statistical hypotheses, enabling the recovery of conventional high-probability or expected generalization guarantees—all without requiring distributional assumptions.

This work investigates the extent to which adversarial attacks reflect a model’s actual robustness under random noise of comparable magnitude, rather than merely characterizing worst-case scenarios. To this end, the authors propose a directional bias perturbation framework governed by a concentration parameter κ, which interpolates smoothly between isotropic noise and adversarial directions. They further introduce a novel attack strategy designed to better approximate realistic statistical noise. Through systematic evaluations on ImageNet and CIFAR-10, the study delineates the conditions under which common adversarial attacks effectively capture noise-induced failure risks, thereby offering both theoretical grounding and practical guidance for safety-oriented robustness evaluation of machine learning models.