SnapPix: Efficient-Coding--Inspired In-Sensor Compression for Edge Vision

📅 2025-04-06
📈 Citations: 0
Influential: 0
📄 PDF
🤖 AI Summary
To address the bottlenecks of limited on-device computation and high transmission energy in edge vision sensing, this paper proposes a novel analog-domain collaborative compression paradigm: encoding exposure (CE) is performed directly at the image sensor frontend to enable spatio-temporal selective sampling. Our key contributions are threefold: (1) the first task-agnostic method for learning efficient encoding sampling patterns; (2) joint optimization of downstream vision models to compensate for pixel-level non-uniform distortions introduced by CE; and (3) a lightweight hardware-enhanced module enabling on-sensor compression. Experiments on action recognition and video reconstruction demonstrate that, at equal frame rates, our approach reduces energy consumption by up to 15.4× compared to state-of-the-art video-based methods—significantly pushing the energy-efficiency frontier for edge vision systems.

Technology Category

Application Category

📝 Abstract
Energy-efficient image acquisition on the edge is crucial for enabling remote sensing applications where the sensor node has weak compute capabilities and must transmit data to a remote server/cloud for processing. To reduce the edge energy consumption, this paper proposes a sensor-algorithm co-designed system called SnapPix, which compresses raw pixels in the analog domain inside the sensor. We use coded exposure (CE) as the in-sensor compression strategy as it offers the flexibility to sample, i.e., selectively expose pixels, both spatially and temporally. SNAPPIX has three contributions. First, we propose a task-agnostic strategy to learn the sampling/exposure pattern based on the classic theory of efficient coding. Second, we co-design the downstream vision model with the exposure pattern to address the pixel-level non-uniformity unique to CE-compressed images. Finally, we propose lightweight augmentations to the image sensor hardware to support our in-sensor CE compression. Evaluating on action recognition and video reconstruction, SnapPix outperforms state-of-the-art video-based methods at the same speed while reducing the energy by up to 15.4x. We have open-sourced the code at: https://github.com/horizon-research/SnapPix.
Problem

Research questions and friction points this paper is trying to address.

Reducing edge energy consumption for remote sensing
Compressing raw pixels in analog domain inside sensor
Addressing pixel-level non-uniformity in CE-compressed images
Innovation

Methods, ideas, or system contributions that make the work stand out.

Analog domain in-sensor pixel compression
Task-agnostic efficient coding exposure pattern
Hardware-lightweight CE compression augmentation
🔎 Similar Papers
No similar papers found.