Closed-Form Robustness Bounds for Second-Order Pruning of Neural Controller Policies

2025-06-29

AI Summary

To address the deployment challenge of deep neural network controllers on resource-constrained embedded systems due to excessive parameter counts, this paper investigates the impact of second-order pruning—specifically Optimal Brain Damage—on closed-loop stability, tracking accuracy, and safety. We propose a forward-propagation analysis framework grounded in local Hessian approximation and the 1-Lipschitz property of ReLU networks. This enables the first derivation of a closed-form, robust upper bound on post-pruning policy output deviation. Crucially, the bound explicitly relates pruning intensity to a user-specified control error threshold, allowing prior determination of the maximum admissible pruning magnitude while guaranteeing pre-deployment performance bounds. Our work bridges deep learning model compression with the reliability requirements of safety-critical control systems, providing theoretical foundations for lightweight, trustworthy deployment of neural controllers.

Technology Category

• Natural Language Processing: Safety and Robustness
• Search and Optimization: Learning to Search
• Intelligent Robots: Learning & Optimization for ROB

Application Category

• Responsible Web: Human-perceived consequences of algorithmic deployment on the web
• Security and Privacy: Large-scale security measurements
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs

Abstract

Deep neural policies have unlocked agile flight for quadcopters, adaptive grasping for manipulators, and reliable navigation for ground robots, yet their millions of weights conflict with the tight memory and real-time constraints of embedded microcontrollers. Second-order pruning methods, such as Optimal Brain Damage (OBD) and its variants, including Optimal Brain Surgeon (OBS) and the recent SparseGPT, compress networks in a single pass by leveraging the local Hessian, achieving far higher sparsity than magnitude thresholding. Despite their success in vision and language, the consequences of such weight removal on closed-loop stability, tracking accuracy, and safety have remained unclear. We present the first mathematically rigorous robustness analysis of second-order pruning in nonlinear discrete-time control. The system evolves under a continuous transition map, while the controller is an $L$-layer multilayer perceptron with ReLU-type activations that are globally 1-Lipschitz. Pruning the weight matrix of layer $k$ replaces $W_k$ with $W_k+δW_k$, producing the perturbed parameter vector $widehatΘ=Θ+δΘ$ and the pruned policy $π(cdot;widehatΘ)$. For every input state $sin X$ we derive the closed-form inequality $ |π(s;Θ)-π(s;widehatΘ)|_2 le C_k(s),|δW_k|_2, $ where the constant $C_k(s)$ depends only on unpruned spectral norms and biases, and can be evaluated in closed form from a single forward pass. The derived bounds specify, prior to field deployment, the maximal admissible pruning magnitude compatible with a prescribed control-error threshold. By linking second-order network compression with closed-loop performance guarantees, our work narrows a crucial gap between modern deep-learning tooling and the robustness demands of safety-critical autonomous systems.

Problem

• Analyzes robustness of second-order pruning in neural controllers
• Derives closed-form bounds for pruning impact on control stability
• Links network compression with safety-critical system performance guarantees

Innovation

• Second-order pruning for neural controller compression
• Closed-form robustness bounds for pruning impact
• Linking compression with closed-loop performance guarantees