Programmable Silicon Retina on Pixel Processor Array

2026-06-06

AI Summary

This work addresses the limitations of conventional dynamic vision sensors (DVS), which respond solely to temporal contrast changes and lack the spatial filtering and gain control mechanisms found in biological retinas, thereby struggling to support tasks such as saliency prediction efficiently. The authors present the first implementation of a multi-stage silicon retina model on the SCAMP-5 programmable pixel processor array, integrating these biologically inspired mechanisms and coupling them with a lightweight FireNet-style network (approximately 100k parameters) for end-to-end evaluation. Experimental results demonstrate that, compared to standard DVS, the proposed approach achieves superior performance in both video intensity reconstruction and saliency prediction, reducing saliency prediction loss by 13% and event rate by 47%, thus validating its capacity for efficient information distillation in bandwidth-constrained edge scenarios.

Abstract

Standard dynamic vision sensors approximate retinal processing by detecting temporal contrast changes, offering high speed and high dynamic range. In this work, we explore whether incorporating additional biologically inspired processing stages - specifically spatial filtering and gain control - can offer advantages for certain downstream tasks such as saliency prediction. We present the first implementation of a multi-stage Silicon Retina model on the SCAMP-5 Pixel Processor Array, along with a GPU-based simulation framework. We evaluate the performance of our model on Video Intensity Reconstruction and Video Saliency Prediction. While the bio-inspired model is less effective at reconstructing absolute intensity frames, it achieves a 13\% reduction in saliency prediction loss in comparison to standard DVS event representation, while reducing the event rate by approximately 47\%. These experiments are obtained using a lightweight $\approx 100$k-parameter FireNet-style network, adapted from event-based reconstruction to saliency prediction. These results suggest that the silicon retina's "information distillation" mechanism can achieve a more efficient representation for downstream neural networks, particularly in bandwidth-constrained edge applications.

Problem

• Silicon Retina
• Dynamic Vision Sensors
• Saliency Prediction
• Event-based Vision
• Biologically Inspired Processing

Innovation

• Silicon Retina
• Pixel Processor Array
• Saliency Prediction
• Event-based Vision
• Biologically Inspired Processing