LiFR v2: Completion-Augmented Event Propagation for High-Rate Dense Prediction

2026-09-22

AI Summary

为解决动态环境中高帧率密集预测受限问题，提出LiFR v2框架，通过事件引导的补全模块和历史检索模块来融合RGB关键帧与事件相机数据，提高预测精度与效率。

Abstract

High-rate dense perception in dynamic environments is limited by the low update rate of RGB cameras, as rapid scene changes can occur between frames. Event cameras offer temporally dense but spatially sparse measurements, complementary to spatially dense RGB observations. Direct fusion cannot fully exploit this complementarity, while event-guided propagation fails on newly appearing or disoccluded regions without valid RGB support. We present LiFR v2, a unified propagation-completion-memory framework for causal anytime and streaming dense prediction from an RGB keyframe and events. LiFR v2 introduces an Event-Guided Completion Module (EGCM) to recover task-relevant representations where propagation is unsupported, and a History Retrieval Module (HRM) to reuse completed representations across successive queries. The framework supports semantic segmentation, monocular depth estimation, and multi-task dense prediction, and we further introduce SHF-Emerge to evaluate rapid object emergence and disocclusion. LiFR v2 achieves 74.37% mIoU on DSEC and 56.13% on SHF-Emerge, improving LiFR-Seg by 1.85 percentage points on the latter, while reducing SHF-Emerge depth RMSE from 1.564 m to 1.118 m over the propagation baseline. It also exceeds 100 FPS for both segmentation and depth, demonstrating accurate and efficient high-rate perception beyond RGB frame rates.

Problem

• high-rate dense prediction
• dynamic environments
• event cameras
• RGB cameras

Innovation

• Event-Guided Completion Module (EGCM)
• History Retrieval Module (HRM)
• high-rate dense prediction
• causal anytime and streaming