A Smoothed Discrepancy Principle for Random Feature Methods and Neural Networks

2026-09-17

AI Summary

研究提出了一种多尺度停止规则，用于非参数回归中的谱正则化方法，并通过随机特征近似减少计算成本，同时选择最优网络宽度以达到最小最大最优学习率。

Abstract

We study data-driven early stopping for spectral regularisation methods in the classical non-parametric regression setting. Building on the discrepancy principle, we propose a multi-scale stopping rule that applies to general kernel estimators and show that, unlike previous approaches, it achieves full adaptivity over all smoothness levels in the well-specified case. A key contribution of our work is an extension based on random feature approximations, which reduces computational cost on large datasets while preserving minimax-optimal statistical guarantees. Our procedure not only selects an optimal stopping time but also provides a fully data-driven choice of the number of random features needed to achieve optimal rates. Through the established connection between random features and neural networks in the neural tangent kernel regime, our method further yields a principled, data-driven recommendation for the network width. We prove that the resulting simultaneously chosen width and stopping time allow neural networks to attain minimax-optimal learning rates without prior knowledge of smoothness or capacity parameters.

Problem

• spectral regularisation
• non-parametric regression
• discrepancy principle
• random feature approximations
• neural networks

Innovation

• multi-scale stopping rule
• random feature approximations
• neural tangent kernel
• data-driven network width selection
• minimax-optimal learning rates