Temporal and Rotational Calibration for Event-Centric Multi-Sensor Systems

2025-08-17

AI Summary

Joint calibration of temporal delay and rotational extrinsics between event cameras and multi-sensor systems remains challenging due to the absence of dedicated calibration targets. Method: We propose a motion-driven, target-free calibration framework that uniquely integrates canonical correlation analysis (CCA) with continuous-time SO(3) nonlinear optimization. Angular velocity is estimated from optical flow derived from event streams; CCA initializes time delay and rotation parameters, which are then jointly refined via continuous-time SO(3) pose modeling. Contribution/Results: The method eliminates reliance on calibration boards while achieving accuracy comparable to board-based approaches—angular error < 0.5° and time-delay error < 1 ms—on both public and in-house datasets. It significantly outperforms pure-CCA baselines, offering high robustness, precision, and practical deployment flexibility.

Technology Category

Computer Vision: Motion & TrackingMachine Learning: Calibration & Uncertainty QuantificationIntelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

Systems and Infrastructure for Web, Mobile and WoT: Experiences and lessons learnt from Web-based algorithms and system deploymentsGraph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphsSearch and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Event cameras generate asynchronous signals in response to pixel-level brightness changes, offering a sensing paradigm with theoretically microsecond-scale latency that can significantly enhance the performance of multi-sensor systems. Extrinsic calibration is a critical prerequisite for effective sensor fusion; however, the configuration that involves event cameras remains an understudied topic. In this paper, we propose a motion-based temporal and rotational calibration framework tailored for event-centric multi-sensor systems, eliminating the need for dedicated calibration targets. Our method uses as input the rotational motion estimates obtained from event cameras and other heterogeneous sensors, respectively. Different from conventional approaches that rely on event-to-frame conversion, our method efficiently estimates angular velocity from normal flow observations, which are derived from the spatio-temporal profile of event data. The overall calibration pipeline adopts a two-step approach: it first initializes the temporal offset and rotational extrinsics by exploiting kinematic correlations in the spirit of Canonical Correlation Analysis (CCA), and then refines both temporal and rotational parameters through a joint non-linear optimization using a continuous-time parametrization in SO(3). Extensive evaluations on both publicly available and self-collected datasets validate that the proposed method achieves calibration accuracy comparable to target-based methods, while exhibiting superior stability over purely CCA-based methods, and highlighting its precision, robustness and flexibility. To facilitate future research, our implementation will be made open-source. Code: https://github.com/NAIL-HNU/EvMultiCalib.

Problem

• Calibrating temporal and rotational parameters for event-centric multi-sensor systems
• Eliminating the need for dedicated calibration targets in sensor fusion
• Estimating angular velocity from event data without event-to-frame conversion

Innovation

• Motion-based calibration without targets
• Angular velocity from normal flow
• Two-step CCA and SO(3) optimization