SCALPEL: Selective Capability Ablation via Low-rank Parameter Editing for Large Language Model Interpretability Analysis

2026-01-12

AI Summary

This work addresses the limited deployability of large language models in high-stakes settings, which stems from insufficient understanding of how internal capabilities are encoded within their parameter space. Existing approaches struggle to precisely characterize the distribution of such capabilities across model parameters. To overcome this, the authors propose SCALPEL, a framework that challenges conventional modular assumptions by revealing that model capabilities can be modeled as low-rank subspaces spanning multiple layers and modules. Leveraging LoRA-based low-rank parameter editing, SCALPEL enables selective ablation of specific capabilities—such as distinguishing correct from incorrect answers—while preserving general language modeling performance. Experiments on the BLiMP benchmark demonstrate the method’s efficacy, offering a fine-grained tool for interpretability through targeted intervention and analysis.

Technology Category

• Natural Language Processing: (Large) Language Models
• Machine Learning: Large Multimodal Models (LMMs)
• Computer Vision: Large Vision Models

Application Category

• Search and Retrieval-Augmented AI: Large language models for search
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Large language models excel across diverse domains, yet their deployment in healthcare, legal systems, and autonomous decision-making remains limited by incomplete understanding of their internal mechanisms. As these models integrate into high-stakes systems, understanding how they encode capabilities has become fundamental to interpretability research. Traditional approaches identify important modules through gradient attribution or activation analysis, assuming specific capabilities map to specific components. However, this oversimplifies neural computation: modules may contribute to multiple capabilities simultaneously, while single capabilities may distribute across multiple modules. These coarse-grained analyses fail to capture fine-grained, distributed capability encoding. We present SCALPEL (Selective Capability Ablation via Low-rank Parameter Editing for Large language models), a framework representing capabilities as low-rank parameter subspaces rather than discrete modules. Our key insight is that capabilities can be characterized by low-rank modifications distributed across layers and modules, enabling precise capability removal without affecting others. By training LoRA adapters to reduce distinguishing correct from incorrect answers while preserving general language modeling quality, SCALPEL identifies low-rank representations responsible for particular capabilities while remaining disentangled from others. Experiments across diverse capability and linguistic tasks from BLiMP demonstrate that SCALPEL successfully removes target capabilities while preserving general capabilities, providing fine-grained insights into capability distribution across parameter space. Results reveal that capabilities exhibit low-rank structure and can be selectively ablated through targeted parameter-space interventions, offering nuanced understanding of capability encoding in LLMs.

Problem

• large language models
• interpretability
• capability encoding
• parameter space
• distributed representation

Innovation

• low-rank parameter editing
• capability ablation
• model interpretability
• LoRA
• distributed capability encoding