🤖 AI Summary
The first layer of CNNs exhibits sensitivity to input translations, leading to unstable parameter learning. Method: We reveal that max-pooling can approximate complex modulus operations—and thus achieve approximate translation invariance—under specific conditions. We propose the first quantitative metric for translation invariance in subsampled convolution followed by max-pooling, and theoretically establish that filter center frequency and orientation are the key determinants of stability. Leveraging the dual-tree complex wavelet packet transform—a discrete Gabor decomposition—we construct a deterministic feature extractor for empirical validation. Contribution/Results: Both theoretical analysis and experiments consistently demonstrate that deliberately designing filters with controlled spectral orientation significantly enhances the translation robustness of pooled feature maps. Our core contribution is a novel, interpretable, and quantifiable framework for analyzing translation stability in CNN first layers, coupled with actionable, frequency-domain filter design principles to improve robustness.
📝 Abstract
This paper focuses on improving the mathematical interpretability of convolutional neural networks (CNNs) in the context of image classification. Specifically, we tackle the instability issue arising in their first layer, which tends to learn parameters that closely resemble oriented band-pass filters when trained on datasets like ImageNet. Subsampled convolutions with such Gabor-like filters are prone to aliasing, causing sensitivity to small input shifts. In this context, we establish conditions under which the max pooling operator approximates a complex modulus, which is nearly shift invariant. We then derive a measure of shift invariance for subsampled convolutions followed by max pooling. In particular, we highlight the crucial role played by the filter's frequency and orientation in achieving stability. We experimentally validate our theory by considering a deterministic feature extractor based on the dual-tree complex wavelet packet transform, a particular case of discrete Gabor-like decomposition.