Orthogonium : A Unified, Efficient Library of Orthogonal and 1-Lipschitz Building Blocks

2026-01-20

AI Summary

Existing implementations of orthogonal and 1-Lipschitz neural network layers are fragmented, functionally limited, and computationally expensive, hindering their adoption in robust deep learning. This work proposes the first unified and efficient PyTorch library that integrates a comprehensive suite of orthogonal and 1-Lipschitz building blocks, supporting standard convolutional operations—including striding, dilation, grouping, and transposed convolutions—while employing optimized parameterizations to rigorously enforce orthogonality and Lipschitz constraints. The library corrects critical errors present in prior implementations, substantially lowers the barrier to entry, enhances training stability on large-scale benchmarks such as ImageNet, reduces computational overhead, and demonstrates correctness and reliability through systematic validation.

Technology Category

Computer Vision: Learning & Optimization for CVMachine Learning: Deep Neural Architectures and Foundation ModelsNatural Language Processing: Safety and Robustness

Application Category

Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphsSystems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applicationsSecurity and Privacy: Data transparency and provenance

Abstract

Orthogonal and 1-Lipschitz neural network layers are essential building blocks in robust deep learning architectures, crucial for certified adversarial robustness, stable generative models, and reliable recurrent networks. Despite significant advancements, existing implementations remain fragmented, limited, and computationally demanding. To address these issues, we introduce Orthogonium , a unified, efficient, and comprehensive PyTorch library providing orthogonal and 1-Lipschitz layers. Orthogonium provides access to standard convolution features-including support for strides, dilation, grouping, and transposed-while maintaining strict mathematical guarantees. Its optimized implementations reduce overhead on large scale benchmarks such as ImageNet. Moreover, rigorous testing within the library has uncovered critical errors in existing implementations, emphasizing the importance of standardized and reliable tools. Orthogonium thus significantly lowers adoption barriers, enabling scalable experimentation and integration across diverse applications requiring orthogonality and robust Lipschitz constraints. Orthogonium is available at https://github.com/deel-ai/orthogonium.

Problem

• orthogonal layers
• 1-Lipschitz
• robust deep learning
• fragmented implementations
• computational overhead

Innovation

• orthogonal layers
• 1-Lipschitz networks
• robust deep learning
• efficient implementation
• PyTorch library