Enhancing Automated Interpretability with Output-Centric Feature Descriptions

2025-01-14

AI Summary

Existing automatic explanation methods predominantly adopt an input-centric paradigm, limiting their ability to characterize causal feature effects on model outputs and handle “dead features” in large language models. This work proposes an output-centric feature attribution paradigm, anchoring explanations at the model’s final output. It enables lightweight, automated causal attribution via steering interventions, vocabulary-decoupled head projections, and token-weight analysis. Crucially, it shifts the explanatory focus from input activations to output effects, explicitly modeling features’ causal contributions to generation outcomes, while supporting dead-feature reactivation and diagnosis. Experiments demonstrate that our method significantly outperforms baselines in output-behavior explanation accuracy. When integrated with both input- and output-centric perspectives, it achieves state-of-the-art performance on both input-relevance and output-faithfulness evaluation metrics.

Technology Category

Machine Learning: Causal LearningNatural Language Processing: (Large) Language ModelsComputer Vision: Large Vision Models

Application Category

Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for rankingUser Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalizationSemantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Automated interpretability pipelines generate natural language descriptions for the concepts represented by features in large language models (LLMs), such as plants or the first word in a sentence. These descriptions are derived using inputs that activate the feature, which may be a dimension or a direction in the model's representation space. However, identifying activating inputs is costly, and the mechanistic role of a feature in model behavior is determined both by how inputs cause a feature to activate and by how feature activation affects outputs. Using steering evaluations, we reveal that current pipelines provide descriptions that fail to capture the causal effect of the feature on outputs. To fix this, we propose efficient, output-centric methods for automatically generating feature descriptions. These methods use the tokens weighted higher after feature stimulation or the highest weight tokens after applying the vocabulary"unembedding"head directly to the feature. Our output-centric descriptions better capture the causal effect of a feature on model outputs than input-centric descriptions, but combining the two leads to the best performance on both input and output evaluations. Lastly, we show that output-centric descriptions can be used to find inputs that activate features previously thought to be"dead".

Problem

• Auto-interpretation
• Large Language Models
• Feature Activation Explanation

Innovation

• Feature Activation Impact
• Enhanced Interpretability
• Improved Performance