Fundamental Limits of Neural Network Sparsification: Evidence from Catastrophic Interpretability Collapse

2026-03-17

AI Summary

This study investigates whether extreme neural network sparsification—specifically, a 90% reduction in activations—triggers an irreversible collapse in mechanistic interpretability. To this end, we introduce an adaptive sparsity scheduling strategy within a hybrid VAE-SAE architecture, combining Top-k and L1 sparsification methods, and systematically evaluate feature viability and interpretability on the dSprites and Shapes3D datasets. Our experiments reveal that while global representation quality remains stable, local interpretability degrades systematically, with up to 90.6% of neurons becoming dead. Moreover, the severity of this collapse intensifies with increasing data complexity. This work establishes, for the first time, a fundamental trade-off between sparsity and interpretability, demonstrating that interpretability collapse is an intrinsic property of the compression process rather than an artifact of specific algorithms or training configurations.

Technology Category

Computer Vision: Interpretability, Explainability, and TransparencyCognitive Modeling & Cognitive Systems: Neural Spike CodingMachine Learning: Learning on the Edge & Model Compression

Application Category

User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalizationSearch and Retrieval-Augmented AI: Web query analysis, representation and understandingGraph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs

Abstract

Extreme neural network sparsification (90% activation reduction) presents a critical challenge for mechanistic interpretability: understanding whether interpretable features survive aggressive compression. This work investigates feature survival under severe capacity constraints in hybrid Variational Autoencoder--Sparse Autoencoder (VAE-SAE) architectures. We introduce an adaptive sparsity scheduling framework that progressively reduces active neurons from 500 to 50 over 50 training epochs, and provide empirical evidence for fundamental limits of the sparsification-interpretability relationship. Testing across two benchmark datasets -- dSprites and Shapes3D -- with both Top-k and L1 sparsification methods, our key finding reveals a pervasive paradox: while global representation quality (measured by Mutual Information Gap) remains stable, local feature interpretability collapses systematically. Under Top-k sparsification, dead neuron rates reach $34.4\pm0.9\%$ on dSprites and $62.7\pm1.3\%$ on Shapes3D at k=50. L1 regularization -- a fundamentally different "soft constraint" paradigm -- produces equal or worse collapse: $41.7\pm4.4\%$ on dSprites and $90.6\pm0.5\%$ on Shapes3D. Extended training for 100 additional epochs fails to recover dead neurons, and the collapse pattern is robust across all tested threshold definitions. Critically, the collapse scales with dataset complexity: Shapes3D (RGB, 6 factors) shows $1.8\times$ more dead neurons than dSprites (grayscale, 5 factors) under Top-k and $2.2\times$ under L1. These findings establish that interpretability collapse under sparsification is intrinsic to the compression process rather than an artifact of any particular algorithm, training duration, or threshold choice.

Problem

• neural network sparsification
• mechanistic interpretability
• feature collapse
• sparse autoencoders
• interpretability limits

Innovation

• sparsification
• interpretability collapse
• adaptive sparsity scheduling
• dead neurons
• mechanistic interpretability