Reachability-Informed Reinforcement Learning for Multi-Impulse Interplanetary Transfers

2026-10-01

AI Summary

This study addresses the disconnect between reinforcement learning (RL) decision-making and orbital geometry, as well as the poor policy generalizability in interplanetary transfers, by proposing the RARL framework. This method innovatively employs a local first-order reachability map as the RL decision interface to constrain waypoint selection, coupled with Lambert reconstruction to ensure dynamically consistent trajectory generation. Experimental results demonstrate that the proposed approach achieves benchmark-level trajectory construction. Furthermore, the multi-state policy maintains zero constraint violations across ten thousand Monte Carlo simulations, significantly enhancing both policy reusability and physical consistency.

Abstract

Reinforcement learning offers the prospect of a reusable sequential decision-making mechanism for spacecraft trajectory design, motivating policy interfaces that connect learned decisions to the underlying maneuver geometry. This paper develops Reachability Analysis-Informed Reinforcement Learning (RARL) for deterministic multi-impulse interplanetary transfers, placing intermediate waypoint selection at the center of the learned decision process. Local first-order reachability maps bounded velocity perturbations into an ellipsoidal set of next-node positions, within which the policy selects its waypoint. Lambert reconstruction then determines the corresponding maneuver to reach this selected waypoint along a dynamically consistent ballistic arc, coupling learned transfer-geometry selection with classical astrodynamics. A terminal two-impulse reconstruction completes the rendezvous, supported by a linear maneuver-demand assessment used for reward shaping. Numerical studies characterize this interface on a two-body Earth-Mars benchmark. Across three independent training runs, RARL achieves a mean maneuver cost of 10.23 km/s, 1.72% above a validated local sequential convex programming reference. Training over dispersed initial states extends policy reuse across a departure family with fixed target state and transfer duration. Each of the three independently trained multi-state policies completes all 10,000 held-out Monte Carlo departures without impulse-cap violations, compared with a mean feasibility rate of 6.49% for single-state policies. This broader sampled feasibility is accompanied by a 0.61% increase in mean nominal maneuver cost, without further training across departures. These results demonstrate that a reachability-informed decision interface supports benchmark-quality trajectory construction and policy reuse across dispersed departure conditions.

Problem

• Reinforcement Learning
• Interplanetary Transfers
• Trajectory Design
• Reachability Analysis
• Multi-Impulse Maneuver

Innovation

• Reachability Analysis
• Reinforcement Learning
• Multi-Impulse Interplanetary Transfers
• Lambert Reconstruction
• Policy Reuse