Agnostic learning in (almost) optimal time via Gaussian surface area

2026-03-06

AI Summary

This work establishes that the efficiency of agnostic learning under Gaussian marginals hinges on whether a concept class can be effectively approximated in the L₁ norm by low-degree polynomials. By refining the L₁ approximation analysis for concept classes with bounded Gaussian surface area, the required polynomial degree to achieve accuracy ε is reduced from O(Γ²/ε⁴) to Õ(Γ²/ε²), nearly matching the theoretical lower bound. The approach extends Feldman et al.’s polynomial approximation technique—originally developed for the Boolean hypercube—to Gaussian space, leveraging connections between Gaussian surface area and polynomial approximation theory. Consequently, it achieves (nearly) optimal agnostic learning time complexity for polynomial threshold functions in the statistical query model, substantially improving upon prior results.

Technology Category

• Machine Learning: Learning Theory
• Search and Optimization: Learning to Search
• Knowledge Representation and Reasoning: Computational Complexity of Reasoning

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Security and Privacy: Large-scale security measurements
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

The complexity of learning a concept class under Gaussian marginals in the difficult agnostic model is closely related to its $L_1$-approximability by low-degree polynomials. For any concept class with Gaussian surface area at most $\Gamma$, Klivans et al. (2008) show that degree $d = O(\Gamma^2 / \varepsilon^4)$ suffices to achieve an $\varepsilon$-approximation. This leads to the best-known bounds on the complexity of learning a variety of concept classes. In this note, we improve their analysis by showing that degree $d = \tilde O (\Gamma^2 / \varepsilon^2)$ is enough. In light of lower bounds due to Diakonikolas et al. (2021), this yields (near) optimal bounds on the complexity of agnostically learning polynomial threshold functions in the statistical query model. Our proof relies on a direct analogue of a construction of Feldman et al. (2020), who considered $L_1$-approximation on the Boolean hypercube.

Problem

• agnostic learning
• Gaussian surface area
• L1-approximation
• polynomial threshold functions
• statistical query model

Innovation

• agnostic learning
• Gaussian surface area
• L1 approximation
• polynomial threshold functions
• statistical query model