CDFlow: Building Invertible Layers with Circulant and Diagonal Matrices

2025-10-29

AI Summary

Normalizing flows suffer from limited expressiveness and high computational cost in invertible linear layers—specifically, expensive Jacobian determinant evaluation and matrix inversion, both scaling as $O(n^3)$. To address this, we propose Circulant-Diagonal Flow (CDFlow), an invertible linear layer constructed via the product of a circulant matrix and a diagonal matrix. This decomposition preserves exact invertibility while reducing parameter complexity to $O(mn)$. Crucially, leveraging the Fast Fourier Transform (FFT), CDFlow computes the Jacobian determinant and inverse matrix in $O(n log n)$ time—exponentially faster than standard dense linear layers. The design is particularly effective for high-dimensional density estimation and generative modeling, especially on data with inherent periodic or quasi-periodic structure. Experiments demonstrate that CDFlow achieves state-of-the-art density estimation performance on natural image benchmarks, with critical operations significantly outperforming existing invertible linear transformations in speed and scalability.

Technology Category

• Machine Learning: Probabilistic Circuits and Graphical Models
• Computer Vision: Other Foundations of Computer Vision
• Knowledge Representation and Reasoning: Computational Complexity of Reasoning

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Responsible Web: Measurement, analysis, and circumvention of Web censorship
• Security and Privacy: Large-scale security measurements

Abstract

Normalizing flows are deep generative models that enable efficient likelihood estimation and sampling through invertible transformations. A key challenge is to design linear layers that enhance expressiveness while maintaining efficient computation of the Jacobian determinant and inverse. We introduce a novel invertible linear layer based on the product of circulant and diagonal matrices. This decomposition reduces parameter complexity from $mathcal{O}(n^2)$ to $mathcal{O}(mn)$ using $m$ diagonal matrices and $m-1$ circulant matrices while still approximating general linear transformations. By leveraging the Fast Fourier Transform, our approach reduces the time complexity of matrix inversion from $mathcal{O}(n^3)$ to $mathcal{O}(mnlog n)$ and that of computing the log-determinant from $mathcal{O}(n^3)$ to $mathcal{O}(mn)$, where $n$ is the input dimension. We build upon this layer to develop Circulant-Diagonal Flow (CDFlow), which achieves strong density estimation on natural image datasets and effectively models data with inherent periodic structure. Furthermore, CDFlow significantly accelerates key operations in normalizing flows, providing practical benefits for scalable generative modeling.

Problem

• Designing invertible linear layers for efficient Jacobian computation
• Reducing parameter complexity in normalizing flow transformations
• Accelerating matrix inversion and determinant calculations in flows

Innovation

• Uses circulant and diagonal matrices for invertible layers
• Reduces parameter complexity from O(n²) to O(mn)
• Leverages Fast Fourier Transform for efficient computations