On the Geometry of Learned Representations in Event-Based Multi-Modal Egomotion Estimation

2026-07-17

AI Summary

This study addresses the problem of ego-motion estimation in event-based multimodal systems by proposing a cross-modal attention architecture that fuses event tensors, inertial measurements, and range signals within an end-to-end learning framework trained in batches. Analysis reveals that the learned representations reside on a low-dimensional manifold aligned with motion variables, and that attention weights adaptively respond to angular velocity excitation and visual reliability, effectively recovering classical geometric observability cues. The work establishes a principled connection between data-driven sensor fusion and analytical motion estimation theory, demonstrating that the learned representations inherently encode physically consistent geometric structures.

Abstract

Classical approaches to event-based egomotion estimation, including those adopted by the top-performing teams of the ELOPE challenge, rely on geometric optimization frameworks such as contrast maximization, homography estimation, or dense optical flow combined with analytic motion inversion. This work investigates the geometric structure that emerges inside a multi-modal network for egomotion estimation. Event tensors, inertial measurements, and range signals are fused through a cross-modal attention architecture and trained in a batch setting. We analyze the latent space geometry and attention dynamics, showing that (i) embeddings lie on low-dimensional manifolds aligned with motion variables, (ii) attention weights adapt with angular excitation and visual reliability, and (iii) the fused representation recovers classical observability cues. These results bridge analytical estimation theory and modern data-driven fusion.

Problem

• egomotion estimation
• event-based vision
• learned representations
• multi-modal fusion
• latent space geometry

Innovation

• event-based vision
• multi-modal fusion
• attention mechanism
• latent space geometry
• egomotion estimation