Towards Ultrafast Depth Sensing Via Active Event-based Stereo Vision

2026-07-26

AI Summary

This work addresses the limitations of conventional frame-based active stereo vision in high-speed scenarios, where depth perception suffers from low efficiency and high latency. The authors propose a novel paradigm—active event-based stereo vision—that integrates binocular event cameras with an infrared 2D structured light projector. They develop a prototype system and introduce ActiveEventNet+, a lightweight neural network featuring a dynamic interaction cost volume, a consistency architecture that efficiently exploits temporal information from event streams, and compact modules tailored for real-time performance. This approach enables the generation of dense disparity maps with low latency. Experimental results demonstrate that the proposed system significantly outperforms existing methods in high-speed settings, achieves substantially reduced computational complexity, and supports real-time processing at 150 FPS.

Abstract

Conventional frame-based imaging for active stereo systems has encountered major challenges in fast-motion scenarios. However, how to design a novel paradigm for ultrafast depth sensing remains an open issue. In this paper, we propose a novel problem setting, namely active event-based stereo vision, which attempts to integrate binocular event cameras and an infrared 2D pattern projector for high-speed dense depth sensing. Technically, we first build a stereo camera prototype system and present a real-world dataset with over 21.5k spatiotemporal synchronized labels at 15 Hz, while also establishing a realistic synthetic dataset with stereo event streams and 23.8k synchronized labels at 20 Hz. Then, we propose ActiveEventNet+, a lightweight yet effective event-based stereo matching neural network that learns to generate high-quality dense disparity maps from stereo event streams with low latency. Our ActiveEventNet+ mainly involves three innovations: incorporating lightweight blocks into event-based stereo matching frameworks, designing a novel cost volume with dynamic interactions between stereo pairs, and presenting an effective temporal consistency architecture to fully use rich temporal cues in event streams. The results show that our ActiveEventNet+ outperforms state-of-the-art methods while significantly reducing computational complexity. Our solution offers superior depth sensing performance compared to conventional frame-based stereo cameras in high-speed scenes. In particular, the lightweight ActiveEventNet enables the prototype system to achieve real-time processing at speeds up to 150 FPS. We believe that this novel active event-based stereo vision paradigm can provide new insights into the design of future high-speed depth sensing camera systems. Our dataset and code can be available at https://github.com/jianing-li/active_event_based_stereo.

Problem

• ultrafast depth sensing
• active stereo vision
• event-based vision
• high-speed scenes
• dense depth estimation

Innovation

• event-based stereo vision
• active depth sensing
• lightweight neural network
• temporal consistency
• cost volume