🤖 AI Summary
This work investigates the dynamic capability evolution mechanisms during large language model (LLM) training, addressing key phenomena including capability formation, biphasic knowledge acquisition, transient learning strategies, and emergent abilities. We propose “developmental interpretability”—a novel framework that models LLM training as a cognitive developmental process, integrating representation probing, causal tracing, and circuit analysis, grounded in cognitive science theory. Our approach systematically reveals three core findings: (1) the progressive construction of computational circuits during training; (2) a biphasic transition in knowledge internalization; and (3) critical phase-transition points underlying emergent capabilities. These results advance mechanistic understanding of LLM capability growth and establish a developmental perspective as a theoretical foundation for AI safety and alignment. The framework enables predictive, intervenable, and trustworthy AI design, offering a principled research roadmap for controllable model development.
📝 Abstract
This review synthesizes the nascent but critical field of developmental interpretability for Large Language Models. We chart the field's evolution from static, post-hoc analysis of trained models to a dynamic investigation of the training process itself. We begin by surveying the foundational methodologies, including representational probing, causal tracing, and circuit analysis, that enable researchers to deconstruct the learning process. The core of this review examines the developmental arc of LLM capabilities, detailing key findings on the formation and composition of computational circuits, the biphasic nature of knowledge acquisition, the transient dynamics of learning strategies like in-context learning, and the phenomenon of emergent abilities as phase transitions in training. We explore illuminating parallels with human cognitive and linguistic development, which provide valuable conceptual frameworks for understanding LLM learning. Finally, we argue that this developmental perspective is not merely an academic exercise but a cornerstone of proactive AI safety, offering a pathway to predict, monitor, and align the processes by which models acquire their capabilities. We conclude by outlining the grand challenges facing the field, such as scalability and automation, and propose a research agenda for building more transparent, reliable, and beneficial AI systems.