Ablating Archetypes: The Stability of Archetypal SAEs is an Artifact of Initialization and Metric Design

2026-06-01

AI Summary

This study addresses the high instability of features extracted by sparse autoencoders (SAEs) across different random seeds, which undermines their reliability as reusable analytical units. Through systematic ablation experiments on prototypical SAEs, the work clarifies the distinction between “stability”—consistency of outcomes—and “stabilization,” referring to intrinsic robustness of the mechanism itself. Employing k-means initialization ablation, cosine similarity metrics, and trajectory diagnostics, the authors demonstrate that the apparent stability of prototypical SAEs primarily stems from fixed initialization rather than inherent architectural stabilization properties; removing this initialization reveals no significant advantage conferred by prototype constraints. Furthermore, the study cautions that cosine similarity evaluations relying on preprocessing are susceptible to geometric distortions, potentially leading to misleading assessments of stability.

Abstract

Dictionary learning with sparse autoencoders (SAEs) produces overcomplete bases from neural network activations that are often interpretable and reduces polysemanticity. However, features from SAEs vary substantially across random seeds -- a problem known as instability. Archetypal SAEs (Fel et al., 2025) were proposed as a general dictionary-learning intervention for more reliable concept extraction, and report more stable dictionaries at the end of training. We demonstrate that the stability claimed by archetypal SAEs is a result of setting identical initialization across multiple runs. Through our analyses, we attempt to clarify two distinct notions in mechanistic interpretability that may be ambiguously used: stability is agreement between two independently trained models, whereas stabilization is the convergence of independently initialized runs toward a common solution. This distinction is critical for mechanistic interpretability of natural language processing (NLP), where feature stability is increasingly used as evidence that SAE features are reusable units of analysis. Experiments from archetypal SAEs share a deterministic k-means decoder initialization, setting inter-run dictionary distance to zero before training begins. When this initialization is removed, the archetypal constraint provides no stabilization advantage in our setting. We further identify a preprocessing-dependent cosine geometry issue that complicates interpretation of endpoint stability metrics. Overall, our study supports the value of studying SAEs within the larger dictionary-learning tradition while showing that stability claims require trajectory diagnostics and initialization ablations.

Problem

• stability
• sparse autoencoders
• mechanistic interpretability
• dictionary learning
• initialization

Innovation

• archetypal SAEs
• stability
• initialization ablation
• dictionary learning
• mechanistic interpretability