Toward a Theory of Generalizability in LLM Mechanistic Interpretability Research

2025-09-26

AI Summary

This study addresses the cross-model generalizability of mechanistic interpretations in large language models (LLMs). To overcome the lack of a systematic theoretical framework for mechanism transfer, we propose the first comprehensive theory of mechanistic generalizability across models, grounded in five analytical dimensions: functionality, development, location, relational structure, and configuration. Using Pythia models (14M–410M parameters) trained with multiple random seeds, we empirically analyze the dynamic evolution of the 1-back attention head. Results demonstrate strong generalizability along the *development* dimension—this mechanism emerges earlier, grows faster, and reaches higher peak activation across models—while exhibiting weak consistency in *location*, underscoring the necessity of multi-dimensional evaluation. Our framework establishes a reproducible, scalable paradigm for cross-model mechanistic analysis, advancing foundational research in LLM interpretability.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Interpretability, Analysis, and Evaluation of NLP Models
• Cognitive Modeling & Cognitive Systems: Analogy

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Research on Large Language Models (LLMs) increasingly focuses on identifying mechanistic explanations for their behaviors, yet the field lacks clear principles for determining when (and how) findings from one model instance generalize to another. This paper addresses a fundamental epistemological challenge: given a mechanistic claim about a particular model, what justifies extrapolating this finding to other LLMs -- and along which dimensions might such generalizations hold? I propose five potential axes of correspondence along which mechanistic claims might generalize, including: functional (whether they satisfy the same functional criteria), developmental (whether they develop at similar points during pretraining), positional (whether they occupy similar absolute or relative positions), relational (whether they interact with other model components in similar ways), and configurational (whether they correspond to particular regions or structures in weight-space). To empirically validate this framework, I analyze "1-back attention heads" (components attending to previous tokens) across pretraining in random seeds of the Pythia models (14M, 70M, 160M, 410M). The results reveal striking consistency in the developmental trajectories of 1-back attention across models, while positional consistency is more limited. Moreover, seeds of larger models systematically show earlier onsets, steeper slopes, and higher peaks of 1-back attention. I also address possible objections to the arguments and proposals outlined here. Finally, I conclude by arguing that progress on the generalizability of mechanistic interpretability research will consist in mapping constitutive design properties of LLMs to their emergent behaviors and mechanisms.

Problem

• Addressing generalizability of mechanistic findings across different LLMs
• Proposing five axes for validating mechanistic claim transferability
• Empirically analyzing attention head consistency across model scales

Innovation

• Proposes five axes for mechanistic generalization
• Empirically validates framework using Pythia models
• Links model design properties to emergent mechanisms