Limits of spectral learning under noise

2026-06-11

AI Summary

This study investigates the stability of spectral methods for coefficient estimation in supervised regression under additive label noise. Focusing on multi-basis sparse spectral representations, the authors derive a closed-form expression for the overlap between noisy and noise-free coefficient vectors via eigen-geometric whitening. Their analysis reveals that the degradation in spectral learning performance is governed by a single intrinsic noise scale and establishes a critical noise threshold beyond which the underlying function structure becomes irrecoverable. The theoretical framework applies to various orthogonal bases—including Fourier, Legendre, Bessel, and Haar—and is corroborated by numerical experiments demonstrating that spectral coefficient estimates become significantly unstable once the noise level exceeds this threshold, thereby impeding accurate recovery of the true function.

Abstract

Learning functional relationships from noisy data is a central problem in scientific inference. Spectral methods approximate unknown functions by expanding them in a basis and estimating the corresponding coefficients from data, but the stability of these coefficients under noise remains poorly understood. Here we study supervised regression with additive label noise using sparse spectral representations across multiple bases and dimensions. We show that noise induces a predictable drift in the learned coefficient vector whose magnitude depends on the effective number of active spectral modes. After whitening the empirical feature geometry, we derive a closed-form expression for the overlap between noisy and noiseless coefficient vectors, revealing a universal degradation curve governed by a single intrinsic noise scale. Numerical experiments across Fourier, Legendre, Bessel, and Haar bases confirm the theoretical prediction. The results demonstrate that spectral learning exhibits a fundamental noise threshold beyond which coefficient estimates become unstable, placing intrinsic limits on recovering functional structure from noisy data.

Problem

• spectral learning
• additive noise
• coefficient stability
• functional recovery
• noise threshold

Innovation

• spectral learning
• label noise
• coefficient stability
• universal degradation
• noise threshold