🤖 AI Summary
Existing methods for estimating the Lipschitz constant of deep neural networks suffer from a trade-off between accuracy and scalability: semidefinite programming (SDP)-based approaches incur prohibitive computational cost and poor scalability, whereas closed-form methods yield overly conservative bounds. This work introduces a novel class of closed-form, scalable Lipschitz upper bounds. By generalizing the parameterization of the LipSDP feasible region and integrating matrix norm inequalities with layer-wise propagation bound optimization, our method enables cooperative utilization of multiple parameterized feasible points—without invoking an SDP solver—for the first time. The approach unifies and generalizes ECLipsE-Fast, achieving significantly tighter bounds and higher computational efficiency on large-scale networks. It supports real-time robustness verification for models with up to hundreds of millions of parameters.
📝 Abstract
Computing tight Lipschitz bounds for deep neural networks is crucial for analyzing their robustness and stability, but existing approaches either produce relatively conservative estimates or rely on semidefinite programming (SDP) formulations (namely the LipSDP condition) that face scalability issues. Building upon ECLipsE-Fast, the state-of-the-art Lipschitz bound method that avoids SDP formulations, we derive a new family of improved scalable Lipschitz bounds that can be combined to outperform ECLipsE-Fast. Specifically, we leverage more general parameterizations of feasible points of LipSDP to derive various closed-form Lipschitz bounds, avoiding the use of SDP solvers. In addition, we show that our technique encompasses ECLipsE-Fast as a special case and leads to a much larger class of scalable Lipschitz bounds for deep neural networks. Our empirical study shows that our bounds improve ECLipsE-Fast, further advancing the scalability and precision of Lipschitz estimation for large neural networks.