Double Descent for Random Fourier Series Models

2026-09-24

AI Summary

This study addresses the lack of rigorous theoretical characterization for the generalization error and double descent phenomenon of random Fourier series models in least squares regression. Leveraging random matrix theory and Stieltjes transforms, combined with spectral analysis of discrete Fourier transform matrices, this work derives exact non-asymptotic risk bounds for the Moore–Penrose estimator. It provides the first rigorous proof of the double descent phenomenon when the parameter-to-sample ratio is fixed, revealing the coupling mechanisms among sample size, dimensionality, and noise that govern regression performance. Numerical simulations further validate the accuracy of these theoretical predictions across both underparameterized and overparameterized regimes.

Abstract

We investigate the least squares linear regression problem with random partial Discrete Fourier Transform (DFT) matrices, providing a rigorous analysis of the model's generalization error. By leveraging tools from random matrix theory, we derive exact non-asymptotic bounds for the risk of the Moore-Penrose estimator, which hold for finite-dimensional problems and reveal the precise dependence on key parameters such as the sample size, dimension, and noise variance. Then we obtain a characterization of the double descent phenomenon in the linear regression context, demonstrating how the risk evolves when the number of parameters $p$ and the number of samples $n$ tend to infinity, with $p/n$ fixed. The analysis relies on applications of the Stieltjes transform for random Fourier matrices, enabling a precise description of the spectral properties of these matrices and their impact on regression performance. To validate our theoretical findings, we present several numerical examples that illustrate the double descent curves. These simulations align closely with our derived bounds, confirming their predictive power in both under-parameterized and over-parameterized regimes.

Problem

• Double Descent
• Linear Regression
• Random Fourier Matrices
• Generalization Error
• Random Matrix Theory

Innovation

• Double Descent
• Random Fourier Series Models
• Random Matrix Theory
• Non-asymptotic Bounds
• Stieltjes Transform