🤖 AI Summary
Detecting overfitting—termed “anti-grokking”—during neural network grokking remains challenging when neither training nor test data are accessible. This work introduces random matrix theory to enable data-free overfitting detection by analyzing deviations of empirical spectral distributions of weight matrices from the Marchenko–Pastur law. The authors propose the concept of “correlation traps” as early indicators of overfitting, identified through such spectral anomalies. By combining weight randomization with Jensen–Shannon divergence measurements, they demonstrate that the number and magnitude of correlation traps strongly correlate with declining test accuracy. Similar patterns are observed across multiple foundational large models, suggesting that correlation traps serve as a universal signature of detrimental overfitting in the absence of data access.
📝 Abstract
Training Neural Networks (NNs) without overfitting is difficult; detecting that overfitting is difficult as well. We present a novel Random Matrix Theory method that detects the onset of overfitting in deep learning models without access to train or test data. For each model layer, we randomize each weight matrix element-wise, $\mathbf{W} \to \mathbf{W}_{\mathrm{rand}}$, fit the randomized empirical spectral distribution with a Marchenko-Pastur distribution, and identify large outliers that violate self-averaging. We call these outliers Correlation Traps. During the onset of overfitting, which we call the "anti-grokking" phase in long-horizon grokking, Correlation Traps form and grow in number and scale as test accuracy decreases while train accuracy remains high. Traps may be benign or may harm generalization; we provide an empirical approach to distinguish between them by passing random data through the trained model and evaluating the JS divergence of output logits. Our findings show that anti-grokking is an additional grokking phase with high train accuracy and decreasing test accuracy, structurally distinct from pre-grokking through its Correlation Traps. More broadly, we find that some foundation-scale LLMs exhibit the same Correlation Traps, indicating potentially harmful overfitting.