Towards Mobile Sensing with Event Cameras on High-mobility Resource-constrained Devices: A Survey

2025-03-29

AI Summary

Event-camera perception on highly mobile, resource-constrained mobile platforms faces three fundamental challenges: noise corruption, semantic sparsity, and data deluge. This paper presents a systematic survey of research from 2014 to 2024, offering the first cross-layer analysis—spanning algorithms, models, and hardware—of core bottlenecks and co-design optimization pathways for on-device event data processing. We propose: (i) a lightweight spiking neural network architecture; (ii) a sparse optical flow–driven online filtering method; (iii) heterogeneous computing acceleration strategies leveraging GPU/NPU synergies; and (iv) a multi-sensor fusion framework. Additionally, we curate an open online resource repository and quantitatively characterize real-time–accuracy trade-offs across key tasks—including visual odometry and object tracking. Our work establishes the first reproducible, software–hardware co-optimized deployment roadmap for brain-inspired perception systems on mobile devices.

Technology Category

• Computer Vision: Motion & Tracking
• Machine Learning: Hardware-aware ML
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Data management and stream processing for Web, mobile and wireless applications
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Security and Privacy: Data transparency and provenance

Abstract

With the increasing complexity of mobile device applications, these devices are evolving toward high mobility. This shift imposes new demands on mobile sensing, particularly in terms of achieving high accuracy and low latency. Event-based vision has emerged as a disruptive paradigm, offering high temporal resolution, low latency, and energy efficiency, making it well-suited for high-accuracy and low-latency sensing tasks on high-mobility platforms. However, the presence of substantial noisy events, the lack of inherent semantic information, and the large data volume pose significant challenges for event-based data processing on resource-constrained mobile devices. This paper surveys the literature over the period 2014-2024, provides a comprehensive overview of event-based mobile sensing systems, covering fundamental principles, event abstraction methods, algorithmic advancements, hardware and software acceleration strategies. We also discuss key applications of event cameras in mobile sensing, including visual odometry, object tracking, optical flow estimation, and 3D reconstruction, while highlighting the challenges associated with event data processing, sensor fusion, and real-time deployment. Furthermore, we outline future research directions, such as improving event camera hardware with advanced optics, leveraging neuromorphic computing for efficient processing, and integrating bio-inspired algorithms to enhance perception. To support ongoing research, we provide an open-source extit{Online Sheet} with curated resources and recent developments. We hope this survey serves as a valuable reference, facilitating the adoption of event-based vision across diverse applications.

Problem

• Achieving high accuracy and low latency in mobile sensing.
• Processing event-based data on resource-constrained mobile devices.
• Addressing noise and lack of semantics in event cameras.

Innovation

• Event-based vision for high-mobility sensing
• Hardware and software acceleration strategies
• Neuromorphic computing for efficient processing