Simplifying Outcomes of Language Model Component Analyses with ELIA

2026-02-20

AI Summary

This work addresses the limited accessibility of existing interpretability tools for language models, which are often too complex for non-expert users to effectively utilize. To bridge this gap, the authors introduce ELIA, an interactive web application that uniquely integrates multiple mechanistic analysis techniques—including Attribution Analysis, Function Vector Analysis, and Circuit Tracing—and innovatively incorporates a vision–language model to automatically generate natural language explanations for complex visualizations. User studies demonstrate that ELIA significantly lowers the barrier to understanding model behavior, with AI-generated explanations effectively mitigating knowledge disparities. Notably, users’ comprehension outcomes show no significant correlation with their prior experience using large language models, underscoring the system’s broad usability and general effectiveness across diverse user backgrounds.

Technology Category

• Humans and AI: Explainable AI (XAI) for Human Understanding
• Natural Language Processing: Interpretability, Analysis, and Evaluation of NLP Models
• Machine Learning: Transparent, Interpretable, Explainable ML

Application Category

• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

While mechanistic interpretability has developed powerful tools to analyze the internal workings of Large Language Models (LLMs), their complexity has created an accessibility gap, limiting their use to specialists. We address this challenge by designing, building, and evaluating ELIA (Explainable Language Interpretability Analysis), an interactive web application that simplifies the outcomes of various language model component analyses for a broader audience. The system integrates three key techniques -- Attribution Analysis, Function Vector Analysis, and Circuit Tracing -- and introduces a novel methodology: using a vision-language model to automatically generate natural language explanations (NLEs) for the complex visualizations produced by these methods. The effectiveness of this approach was empirically validated through a mixed-methods user study, which revealed a clear preference for interactive, explorable interfaces over simpler, static visualizations. A key finding was that the AI-powered explanations helped bridge the knowledge gap for non-experts; a statistical analysis showed no significant correlation between a user's prior LLM experience and their comprehension scores, suggesting that the system reduced barriers to comprehension across experience levels. We conclude that an AI system can indeed simplify complex model analyses, but its true power is unlocked when paired with thoughtful, user-centered design that prioritizes interactivity, specificity, and narrative guidance.

Problem

• mechanistic interpretability
• Large Language Models
• accessibility gap
• non-experts
• model analysis

Innovation

• Explainable AI
• Natural Language Explanations
• Interactive Visualization
• Mechanistic Interpretability
• Vision-Language Models