Application of Deep Reinforcement Learning to UAV Swarming for Ground Surveillance

2023-10-27

Italian National Conference on Sensors

AI Summary

This study addresses the challenges of collaborative efficiency and robustness in multi-UAV swarm operations for ground security surveillance—specifically target search, localization, and persistent tracking. We propose a hierarchical hybrid AI architecture: a centralized swarm controller orchestrates high-level task allocation, while each UAV executes role-specific sub-agents for search, localization, and tracking. Notably, we introduce the first task-role-differentiated deployment of Proximal Policy Optimization (PPO) reinforcement learning models across the swarm and design a surveillance-oriented evaluation metric suite. Simulation results demonstrate efficient area coverage, rapid target acquisition, high tracking stability (interruption rate < 2%), and low-latency response. The framework balances technical sophistication with pedagogical interpretability, making it suitable for AI and robotics education at the secondary-school level.

Technology Category

• Search and Optimization: Learning to Search
• Humans and AI: Planning and Decision Support for Human-Machine Teams
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Security and Privacy: Security and privacy of machine learning and AI applications
• Search and Retrieval-Augmented AI: Vertical and domain-specific search
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

This paper summarizes in depth the state of the art of aerial swarms, covering both classical and new reinforcement-learning-based approaches for their management. Then, it proposes a hybrid AI system, integrating deep reinforcement learning in a multi-agent centralized swarm architecture. The proposed system is tailored to perform surveillance of a specific area, searching and tracking ground targets, for security and law enforcement applications. The swarm is governed by a central swarm controller responsible for distributing different search and tracking tasks among the cooperating UAVs. Each UAV agent is then controlled by a collection of cooperative sub-agents, whose behaviors have been trained using different deep reinforcement learning models, tailored for the different task types proposed by the swarm controller. More specifically, proximal policy optimization (PPO) algorithms were used to train the agents’ behavior. In addition, several metrics to assess the performance of the swarm in this application were defined. The results obtained through simulation show that our system searches the operation area effectively, acquires the targets in a reasonable time, and is capable of tracking them continuously and consistently.

Problem

• Advanced Learning Methods
• Drone Teams Efficiency
• Ground Situation Awareness

Innovation

• Hybrid Intelligence System
• PPO Algorithm
• Drone Swarm Optimization