Double Descent and Overparameterization in Particle Physics Data

2025-09-01

AI Summary

This study systematically investigates the “double descent” phenomenon in particle physics data analysis—the non-monotonic reduction of generalization error beyond the interpolation threshold—challenging classical bias-variance trade-off theory. Method: Using high-dimensional real-world particle collision data (e.g., LHC simulations) and diverse deep neural network architectures, we quantitatively analyze how model size, training dataset scale, and regularization strength shape the generalization error curve. Results: Under specific data distributions and optimization configurations, generalization error exhibits a pronounced descent after the interpolation threshold, violating conventional statistical learning expectations. This improvement stems from implicit regularization effects synergizing with the intrinsic high-dimensional geometric structure of physically meaningful feature spaces. Our findings provide both theoretical justification and practical guidance for deploying large-scale models in high-energy physics analyses, establishing foundational insights into overparameterized learning in scientific domains.

Technology Category

• Machine Learning: Learning with Manifolds
• Natural Language Processing: Learning & Optimization for NLP
• Search and Optimization: Learning to Search

Application Category

• User Modeling, Personalization and Recommendation: Practical large-scale studies of user experience
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods

Abstract

Recently, the benefit of heavily overparameterized models has been observed in machine learning tasks: models with enough capacity to easily cross the emph{interpolation threshold} improve in generalization error compared to the classical bias-variance tradeoff regime. We demonstrate this behavior for the first time in particle physics data and explore when and where `double descent' appears and under which circumstances overparameterization results in a performance gain.

Problem

• Demonstrating double descent behavior in particle physics data
• Exploring when and where double descent phenomena appears
• Identifying circumstances where overparameterization improves performance

Innovation

• Overparameterized models crossing interpolation threshold
• Demonstrating double descent in particle physics data
• Exploring performance gains from overparameterization circumstances