SAEExplainer: Interpreting SAE Features with Activation-Guided Preference Optimization

2026-06-07

AI Summary

This work addresses the limited interpretability of sparse autoencoder (SAE) features stemming from the absence of feedback mechanisms in existing open-loop approaches, which hinders the optimization of explanation quality. To overcome this, the paper introduces SAEExplainer, a novel framework that incorporates a closed-loop feedback mechanism for the first time. By integrating activation-guided preference optimization, iterative bootstrapped training, and causal validation, SAEExplainer enables self-correction and continuous refinement of feature interpretations. The proposed method substantially reduces explanatory hallucinations and strengthens the identification of causal activation patterns. Empirical evaluations demonstrate that SAEExplainer outperforms current baselines across multiple metrics, achieving particularly notable gains in causal trigger accuracy and discriminative activation fidelity.

Abstract

Although Sparse Autoencoders (SAEs) have mitigated the opacity of large language models (LLMs) by decomposing dense representations into sparse features, explaining these features still remains a central challenge. Current explanation methods, however, typically operate within an open-loop paradigm, failing to leverage mechanistic feedback for further refinement. In this paper, we propose SAEExplainer, a training framework utilizes activation scores as an objective reward signal to train the model for self-correction and iterative bootstrapping. By iteratively verifying and correcting foundational explanations through a two-round optimization process, SAEExplainer achieves continuous improvement in its explanatory capabilities. This mechanism significantly reduces explanation hallucinations and reinforces causal triggering patterns. Extensive experiments demonstrate our approach improves upon established baselines across most metrics, especially in causal triggering and discriminative activation.

Problem

• Sparse Autoencoders
• feature interpretation
• explanation hallucination
• causal triggering
• mechanistic interpretability

Innovation

• Sparse Autoencoders
• Activation-Guided Optimization
• Interpretability
• Preference Optimization
• Causal Triggers