Cooperative Bearing-Only Target Pursuit via Multiagent Reinforcement Learning: Design and Experiment

2025-03-11

AI Summary

This work addresses multi-robot cooperative tracking of unknown targets using only visual bearing measurements and heterogeneous ground vehicles in complex environments. To tackle filter instability arising from bearing measurement nonlinearity and angular representation singularities, we propose a Uniform Bearing Information Filter (UBIF). To address limited field-of-view (FoV), heterogeneous dynamics, and the simulation-to-reality gap, we introduce a novel spectral-normalized multi-agent reinforcement learning framework—a MAPPO variant—that jointly models FoV-aware perception and learns low-level control gains, enabling zero-shot sim-to-real transfer. Experiments demonstrate high stability on both simulated and real-world AGV platforms, significantly improving occlusion recovery speed and cooperative robustness. Crucially, the method deploys directly to physical hardware without fine-tuning.

Technology Category

• Intelligent Robots: State Estimation
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Multiagent Systems: Adversarial Agents

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

This paper addresses the multi-robot pursuit problem for an unknown target, encompassing both target state estimation and pursuit control. First, in state estimation, we focus on using only bearing information, as it is readily available from vision sensors and effective for small, distant targets. Challenges such as instability due to the nonlinearity of bearing measurements and singularities in the two-angle representation are addressed through a proposed uniform bearing-only information filter. This filter integrates multiple 3D bearing measurements, provides a concise formulation, and enhances stability and resilience to target loss caused by limited field of view (FoV). Second, in target pursuit control within complex environments, where challenges such as heterogeneity and limited FoV arise, conventional methods like differential games or Voronoi partitioning often prove inadequate. To address these limitations, we propose a novel multiagent reinforcement learning (MARL) framework, enabling multiple heterogeneous vehicles to search, localize, and follow a target while effectively handling those challenges. Third, to bridge the sim-to-real gap, we propose two key techniques: incorporating adjustable low-level control gains in training to replicate the dynamics of real-world autonomous ground vehicles (AGVs), and proposing spectral-normalized RL algorithms to enhance policy smoothness and robustness. Finally, we demonstrate the successful zero-shot transfer of the MARL controllers to AGVs, validating the effectiveness and practical feasibility of our approach. The accompanying video is available at https://youtu.be/HO7FJyZiJ3E.

Problem

• Estimating target state using only bearing information in multi-robot pursuit.
• Developing a MARL framework for heterogeneous vehicles to pursue targets in complex environments.
• Bridging the sim-to-real gap for effective deployment of MARL controllers on AGVs.

Innovation

• Uniform bearing-only filter for stable 3D measurements
• Multiagent reinforcement learning for heterogeneous vehicle control
• Spectral-normalized RL for smooth, robust policy transfer