Generalization Dynamics of LM Pre-training

2026-09-27

AI Summary

This study addresses the observation that the generalization capability of language models during pretraining does not improve monotonically, but instead oscillates frequently between rote memorization and intelligent reasoning. To investigate this, we construct an evaluation suite to identify and define the "mode jumping" phenomenon, modeling it as a circuit competition problem under capacity constraints. We propose a theoretical framework for capacity allocation, wherein data windows govern circuit competition, and integrate intermediate checkpoint selection with pretraining data selection strategies to monitor and control generalization dynamics. Our findings challenge the conventional assumption of stable model maturation by demonstrating that intermediate checkpoints can exhibit superior reasoning and alignment capabilities compared to the final model. Furthermore, we show that strategic data selection effectively stabilizes the generalization process throughout pretraining.

Abstract

People typically assume that LMs stably mature from pattern-matching parrots to generalizable intelligence during pre-training. We build a toy eval suite and show this mental model is wrong: throughout pre-training, LMs frequently and suddenly hop between parrot-like and intelligence-like computations. We call this mode-hopping. Across our suite, LMs suddenly latch onto memorized or in-context patterns instead of in-context learning, use System 1 instead of System 2 thinking, pick up what sounds true instead of what is true, fail at multi-hop persona QA, out-of-context reasoning, and emergent misalignment -- then just as suddenly revert and generalize. Mode-hopping is not explained by standard optimization dynamics: it is locally stable and cannot be fixed by checkpoint averaging. We instead think of it as a capacity allocation problem: in a capacity-bounded model, generalizable circuits must compete with the shallow ones learned early in training, and the data in each pre-training window may decide which circuits win. Our suite provides a new efficient lens on generalization. We demonstrate two concrete applications: (i) select intermediate pre-training checkpoints that strongly generalize reasoning and alignment, better than the final pre- or mid-training checkpoints, and (ii) select pre-training data that controls and stabilizes generalization dynamics.

Problem

• Generalization Dynamics
• Mode-hopping
• Language Model Pre-training
• Capacity Allocation

Innovation

• Mode-hopping
• Capacity allocation
• Generalization dynamics
• Checkpoint selection
• Pre-training data curation