The Functional Structure of Post-Compression Recovery in Low-Rank LLMs

2026-10-04

AI Summary

This study addresses the heterogeneous recovery behaviors and unclear functional-structural evolution in low-rank compressed large language models. It introduces the concept of "recovery pressure" to characterize compression-induced changes in functional structure, alongside standardized forward-backward feature analysis methods and module-level dynamic tracking algorithms. The research demonstrates that endpoint recovery pressure effectively predicts subsequent responses, revealing non-uniform modular reorganization patterns during recovery and distributed response mechanisms triggered by local updates. This work establishes a unified functional representation framework across heterogeneous methods, opens a new pathway for forward feature extraction without backpropagation, and provides a complementary functional perspective beyond scalar loss for model evaluation.

Abstract

Different low-rank compression methods can produce compressed LLMs that respond differently to the same post-compression recovery procedure, and relative advantages observed between methods at the compression endpoint may shrink, grow, or even reverse after recovery. We ask whether this recovery heterogeneity reflects functional structure beyond scalar loss evolution, and how that structure evolves throughout recovery. Our results establish that this heterogeneity reflects a reproducible compression-induced functional structure, which we formalize as recovery pressure. To characterize this structure consistently throughout recovery, we develop a standardized functional characterization within each backbone that is applicable across heterogeneous low-rank methods. The primary backward characterization reveals reproducible module-wise structure across independent probes, while a complementary forward-only characterization recovers related structure without loss or backpropagation. We further find that recovery pressure measured at the endpoint is associated with subsequent recovery response; during recovery, its module-wise structure is reorganized non-uniformly, and localized updates induce distributed responses beyond directly updated modules. Further evidence indicates that tracking this evolving structure provides a complementary functional view of recovery progress alongside scalar loss.

Problem

• Low-Rank Compression
• Post-Compression Recovery
• Large Language Models
• Recovery Heterogeneity
• Functional Structure

Innovation

• Low-Rank Compression
• Recovery Pressure
• Functional Characterization
• Post-Compression Recovery
• Large Language Models