Event-Frame Fusion for Inter-Frame Segmentation via Event-Guided Motion

2026-09-18

AI Summary

为解决传统相机在运动模糊、延迟等问题，提出结合事件相机和帧相机的混合视觉架构，通过运动估计和语义分割提高分割精度和效率。

Abstract

Autonomous navigation requires precise and efficient semantic segmentation, yet existing frame-based approaches remain limited by motion blur, glare, latency, and the low temporal resolution (20-30 FPS) of conventional cameras, which leads to information loss between frames. Event cameras have emerged as an alternative sensing modality, capturing intensity changes asynchronously with high temporal resolution, high dynamic range, and sparse outputs. However, event-based algorithms still fall short of frame-based ones in accuracy, as most segmentation methods are designed for dense frame data. To overcome these limitations, we propose a hybrid vision architecture that combines conventional frame-based and event-based cameras. The system integrates two complementary components: (1) a compact Spiking Neural Network (SNN) with 42k parameters for motion estimation, and (2) a lightweight event-driven SNN with 0.84M parameters for frame-based semantic segmentation, which interpolates motion between frames to refine segmentation results. By predicting inter-frame segmentations, the framework achieves segmentation rates of up to 500 Hz with an energy consumption below 1.87 mJ per inference, while maintaining real-time GPU execution at frequencies up to 200 Hz. Additionally, our approach compensates for information loss in frames affected by blur or overexposure, enabling more robust perception in challenging conditions.

Problem

• autonomous navigation
• semantic segmentation
• event cameras
• motion blur
• low temporal resolution

Innovation

• Event-Frame Fusion
• Spiking Neural Network (SNN)
• Inter-Frame Segmentation
• Hybrid Vision Architecture
• Real-Time Semantic Segmentation