Towards Agile Vision-Based Multi-UAV Flight: Revisiting State Estimation

2026-09-30

AI Summary

This study addresses the problem of high-order state delays in vision-based estimation that hinder agile multi-UAV flight. We propose a linear thrust-constrained Kalman filter incorporating tilt measurements, which leverages tilt information to anticipate thrust direction changes, thereby accelerating state convergence and eliminating structural delays. By integrating visual detection with nonlinear model predictive control (NMPC), the method enables efficient coordination and obstacle avoidance. Experimental results demonstrate that the proposed approach reduces velocity and acceleration estimation errors by 40% and 57%, respectively, while supporting stable tracking under accelerations exceeding 2g. These findings indicate a significant improvement in the agile flight performance of multi-UAV systems.

Abstract

Agile multi-UAV flight requires accurate and low-latency onboard estimation of the kinematic states of neighboring UAVs for collision avoidance, motion coordination, etc. Most vision-based approaches rely on position-only measurements, inferring velocity and acceleration indirectly from displacement. We show that this introduces a fixed structural delay in the estimation of higher-order states, which limits the achievable agility. To address this, we propose to integrate tilt measurements, provided by a state-of-the-art visual detector, which inform about the thrust direction of co-planar multirotor UAVs. We benchmark four position-only and five pose-aware estimators, including a novel formulation of a linear thrust-constraining Kalman filter, on two real-world and one high-fidelity photorealistic simulated dataset over different levels of agility (3-21 m/s^2). In our setup, pose-aware estimation consistently reduces the average velocity and acceleration estimation errors by 40% and 57% across the three datasets with the proposed KF formulation outperforming the other estimators. Position-only filters exhibit a constant ~300 ms delay in acceleration step response independent of agility, whereas the tilt-constrained estimators operate near the physical response limit given by the camera frame-rate by observing the change in thrust direction before the displacement accumulates. In a closed-loop leader-follower simulated experiment with NMPC control, position-only estimation of the leader's state fails to facilitate stable hovering of the follower, while the proposed estimator enables tracking of lateral maneuvers exceeding 2g of acceleration.

Problem

• multi-UAV
• vision-based state estimation
• agile flight
• structural delay
• pose-aware estimation

Innovation

• Vision-based state estimation
• Multi-UAV flight
• Tilt-constrained Kalman filter
• Pose-aware estimation
• Agile tracking