🤖 AI Summary
This study addresses the challenge of real-time path planning for autonomous vehicles in environments containing circular no-fly threat zones, where conventional optimal control methods suffer from high computational complexity. To overcome this limitation, the authors propose a reinforcement learning framework based on Deep Deterministic Policy Gradient (DDPG), employing an Actor-Critic architecture and a carefully designed reward function to enable direct mapping from states to actions, thereby rapidly generating safe and feasible trajectories. Notably, the approach innovatively leverages DDPG to construct a “feasibility set” for path planning, offering a priori judgment of task realizability before execution. Simulation results demonstrate that, within this feasibility set, the method achieves significantly higher computational efficiency than pseudospectral optimal control, making it suitable for real-time applications—albeit at the cost of global optimality—while effectively avoiding infeasible regions.
📝 Abstract
Path-planning for autonomous vehicles in threat-laden environments is a fundamental challenge. While traditional optimal control methods can find ideal paths, the computational time is often too slow for real-time decision-making. To solve this challenge, we propose a method based on Deep Deterministic Policy Gradient (DDPG) and model the threat as a simple, circular `no-go' zone. A mission failure is claimed if the vehicle enters this `no-go' zone at any time or does not reach a neighborhood of the destination. The DDPG agent is trained to learn a direct mapping from its current state (position and velocity) to a series of feasible actions that guide the agent to safely reach its goal. A reward function and two neural networks, critic and actor, are used to describe the environment and guide the control efforts. The DDPG trains the agent to find the largest possible set of starting points (``feasible set'') wherein a safe path to the goal is guaranteed. This provides critical information for mission planning, showing beforehand whether a task is achievable from a given starting point, assisting pre-mission planning activities. The approach is validated in simulation. A comparison between the DDPG method and a traditional optimal control (pseudo-spectral) method is carried out. The results show that the learning-based agent may produce effective paths while being significantly faster, making it a better fit for real-time applications. However, there are areas (``infeasible set'') where the DDPG agent cannot find paths to the destination, and the paths in the feasible set may not be optimal. These preliminary results guide our future research: (1) improve the reward function to enlarge the DDPG feasible set, (2) examine the feasible set obtained by the pseudo-spectral method, and (3) investigate the arc-search IPM method for the path planning problem.