Technology Category
Computer Vision: Diffusion Models for VisionMachine Learning: Learning on the Edge & Model CompressionSearch and Optimization: Learning to Search
What and Where to Adapt: Structure-Semantics Co-Tuning for Machine Vision Compression via Synergistic Adapters
🤖 AI Summary
This work addresses the limitation of existing image compression methods that neglect the joint optimization of statistical and semantic information in entropy models when adapting pretrained codecs, thereby constraining the effectiveness of parameter-efficient fine-tuning. To overcome this, the authors propose S2-CoT, a structure–semantics co-tuning framework that systematically analyzes and coordinates adapter type and placement. Specifically, they introduce a Structure-Fidelity Adapter (SFA) for the codec and a Semantic Context Adapter (SCA) for the entropy model, enabling dual-adapter joint optimization through parameter-efficient fine-tuning, spatial–frequency feature fusion, and channel-wise context modeling. Evaluated on four mainstream codecs, S2-CoT achieves performance comparable to full fine-tuning using only a minimal number of trainable parameters, significantly enhancing compression efficiency for machine vision tasks and establishing new state-of-the-art results.
Technology Category
Application Category
📝 Abstract
Parameter-efficient fine-tuning of pre-trained codecs is a promising direction in image compression for human and machine vision. While most existing works have primarily focused on tuning the feature structure within the encoder-decoder backbones, the adaptation of the statistical semantics within the entropy model has received limited attention despite its function of predicting the probability distribution of latent features. Our analysis reveals that naive adapter insertion into the entropy model can lead to suboptimal outcomes, underscoring that the effectiveness of adapter-based tuning depends critically on the coordination between adapter type and placement across the compression pipeline. Therefore, we introduce Structure-Semantics Co-Tuning (S2-CoT), a novel framework that achieves this coordination via two specialized, synergistic adapters: the Structural Fidelity Adapter (SFA) and the Semantic Context Adapter (SCA). SFA is integrated into the encoder-decoder to preserve high-fidelity representations by dynamically fusing spatial and frequency information; meanwhile, the SCA adapts the entropy model to align with SFA-tuned features by refining the channel context for more efficient statistical coding. Through joint optimization, S2-CoT turns potential performance degradation into synergistic gains, achieving state-of-the-art results across four diverse base codecs with only a small fraction of trainable parameters, closely matching full fine-tuning performance. Code is available at https://github.com/Brock-bit4/S2-CoT.
Problem
Research questions and friction points this paper is trying to address.
image compression
parameter-efficient fine-tuning
entropy model
adapter coordination
machine vision
Innovation
Methods, ideas, or system contributions that make the work stand out.
Structure-Semantics Co-Tuning
Parameter-Efficient Fine-Tuning
Entropy Model Adaptation
Synergistic Adapters
Machine Vision Compression
🔎 Similar Papers
No similar papers found.
