SlowFast-SCI: Slow-Fast Deep Unfolding Learning for Spectral Compressive Imaging

2025-09-19

AI Summary

Existing deep unfolding methods for spectral compressive imaging (SCI) rely on slow pre-training and struggle to rapidly adapt to unseen optical configurations, resulting in poor out-of-distribution generalization and inefficient inference. To address this, we propose SlowFast-SCI—a dual-speed deep unfolding framework that introduces test-time adaptation to SCI reconstruction for the first time, enabling lightweight, backpropagation-free online optimization of the backbone network. Our approach integrates imaging-guided knowledge distillation, dual-domain loss, self-supervised test-time fine-tuning, and a modular adaptation architecture, ensuring compatibility with diverse deep unfolding networks. Experiments demonstrate that SlowFast-SCI achieves up to 5.79 dB PSNR improvement on out-of-distribution cameras, accelerates adaptation by 4×, and reduces both parameter count and computational cost by over 70%. The method significantly enhances robustness, inference efficiency, and deployment flexibility.

Technology Category

• Computer Vision: Learning & Optimization for CV
• Machine Learning: Learning on the Edge & Model Compression
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs

Abstract

Humans learn in two complementary ways: a slow, cumulative process that builds broad, general knowledge, and a fast, on-the-fly process that captures specific experiences. Existing deep-unfolding methods for spectral compressive imaging (SCI) mirror only the slow component-relying on heavy pre-training with many unfolding stages-yet they lack the rapid adaptation needed to handle new optical configurations. As a result, they falter on out-of-distribution cameras, especially in bespoke spectral setups unseen during training. This depth also incurs heavy computation and slow inference. To bridge this gap, we introduce SlowFast-SCI, a dual-speed framework seamlessly integrated into any deep unfolding network beyond SCI systems. During slow learning, we pre-train or reuse a priors-based backbone and distill it via imaging guidance into a compact fast-unfolding model. In the fast learning stage, lightweight adaptation modules are embedded within each block and trained self-supervised at test time via a dual-domain loss-without retraining the backbone. To the best of our knowledge, SlowFast-SCI is the first test-time adaptation-driven deep unfolding framework for efficient, self-adaptive spectral reconstruction. Its dual-stage design unites offline robustness with on-the-fly per-sample calibration-yielding over 70% reduction in parameters and FLOPs, up to 5.79 dB PSNR improvement on out-of-distribution data, preserved cross-domain adaptability, and a 4x faster adaptation speed. In addition, its modularity integrates with any deep-unfolding network, paving the way for self-adaptive, field-deployable imaging and expanded computational imaging modalities. Code and models are available at https://github.com/XuanLu11/SlowFast-SCI.

Problem

• Existing SCI methods lack rapid adaptation for new optical configurations
• Deep unfolding networks falter on out-of-distribution spectral cameras
• Current approaches suffer from heavy computation and slow inference speeds

Innovation

• Dual-speed framework combines slow pre-training with fast adaptation
• Lightweight modules enable self-supervised test-time adaptation via dual-domain loss
• Modular design integrates with any deep-unfolding network for efficiency