Recurrent Auto-Encoders for Enhanced Deep Reinforcement Learning in Wilderness Search and Rescue Planning

2025-02-26

AI Summary

To address the path-planning challenge in wilderness search-and-rescue—specifically, maximizing information acquisition rate within strict time constraints—this paper proposes an end-to-end perception-decision framework integrating a Recurrent Autoencoder (RAE) with Deep Reinforcement Learning (DRL). The method innovatively leverages RAE’s temporal compression capability to jointly optimize SAC/PPO policies, eliminating redundant independent modeling of observations. This co-design reduces model parameters by over 80% (to <20% of baseline) and cuts training overhead by 75% (to one-quarter of baseline). Evaluated in large-scale, unstructured environments, the approach significantly enhances real-time UAV search efficiency and policy generalizability. It achieves state-of-the-art search success rates on complex, nonlinear tasks, establishing a scalable, information-value-driven paradigm for resource-constrained search operations.

Technology Category

Search and Optimization: Learning to SearchConstraint Satisfaction and Optimization: SearchPlanning, Routing, and Scheduling: Planning with Markov Models (MDPs, POMDPs)

Application Category

Search and Retrieval-Augmented AI: Efficiency and scalability of Web search enginesUser Modeling, Personalization and Recommendation: ML for personalized search and recommendationsResponsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

Wilderness search and rescue operations are often carried out over vast landscapes. The search efforts, however, must be undertaken in minimum time to maximize the chance of survival of the victim. Whilst the advent of cheap multicopters in recent years has changed the way search operations are handled, it has not solved the challenges of the massive areas at hand. The problem therefore is not one of complete coverage, but one of maximizing the information gathered in the limited time available. In this work we propose that a combination of a recurrent autoencoder and deep reinforcement learning is a more efficient solution to the search problem than previous pure deep reinforcement learning or optimisation approaches. The autoencoder training paradigm efficiently maximizes the information throughput of the encoder into its latent space representation which deep reinforcement learning is primed to leverage. Without the overhead of independently solving the problem that the recurrent autoencoder is designed for, it is more efficient in learning the control task. We further implement three additional architectures for a comprehensive comparison of the main proposed architecture. Similarly, we apply both soft actor-critic and proximal policy optimisation to provide an insight into the performance of both in a highly non-linear and complex application with a large observation Results show that the proposed architecture is vastly superior to the benchmarks, with soft actor-critic achieving the best performance. This model further outperformed work from the literature whilst having below a fifth of the total learnable parameters and training in a quarter of the time.

Problem

• Maximizing information in wilderness search
• Efficient deep reinforcement learning solution
• Recurrent autoencoder enhances search efficiency

Innovation

• Recurrent autoencoder enhances information throughput
• Deep reinforcement learning leverages latent space
• Soft actor-critic achieves superior performance