A Joint Visual Compression and Perception Framework for Neuralmorphic Spiking Camera
🤖 AI Summary
Neuromorphic event cameras offer ultra-high temporal resolution but generate sparse binary spike streams, incurring substantial storage and transmission overhead. To address this, we propose the Spike Coding for Intelligence (SCI) paradigm—a joint compression-and-sensing framework. SCI features a bio-inspired dual-path network that separately models spatial semantics and motion dynamics; incorporates optical-flow-guided deformable alignment, motion-vector consistency refinement, and multi-dynamics temporal regression for robust spike sequence encoding. Crucially, SCI preserves downstream task performance while achieving high-fidelity, compact representations. Experiments demonstrate that SCI reduces BD-rate by 17.25% on average over state-of-the-art codecs, improves image classification accuracy by 4.3% relative to SpiReco, lowers encoder computational complexity by 88.26%, and cuts inference latency by 42.41%.
Application Category
📝 Abstract
The advent of neuralmorphic spike cameras has garnered significant attention for their ability to capture continuous motion with unparalleled temporal resolution.However, this imaging attribute necessitates considerable resources for binary spike data storage and transmission.In light of compression and spike-driven intelligent applications, we present the notion of Spike Coding for Intelligence (SCI), wherein spike sequences are compressed and optimized for both bit-rate and task performance.Drawing inspiration from the mammalian vision system, we propose a dual-pathway architecture for separate processing of spatial semantics and motion information, which is then merged to produce features for compression.A refinement scheme is also introduced to ensure consistency between decoded features and motion vectors.We further propose a temporal regression approach that integrates various motion dynamics, capitalizing on the advancements in warping and deformation simultaneously.Comprehensive experiments demonstrate our scheme achieves state-of-the-art (SOTA) performance for spike compression and analysis.We achieve an average 17.25% BD-rate reduction compared to SOTA codecs and a 4.3% accuracy improvement over SpiReco for spike-based classification, with 88.26% complexity reduction and 42.41% inference time saving on the encoding side.
Problem
Research questions and friction points this paper is trying to address.
Compress binary spike data for storage and transmission efficiency
Optimize spike sequences for both bit-rate and task performance
Enhance spike-based classification accuracy and reduce computational complexity
Innovation
Methods, ideas, or system contributions that make the work stand out.
Spike Coding for Intelligence (SCI) compression
Dual-pathway architecture for spatial and motion processing
Temporal regression integrating motion dynamics
🔎 Similar Papers
No similar papers found.
