GPG-HT: Generalized Policy Gradient with History-Aware Decision Transformer for Probabilistic Path Planning

2025-08-24

AI Summary

Reliable path planning in urban traffic is challenged by the coexistence of stochasticity and spatiotemporal dependencies. Method: This paper proposes a novel approach integrating a history-aware Decision Transformer with a Generalized Policy Gradient (GPG) framework. It is the first to adapt the Decision Transformer to traffic routing—explicitly modeling long-range temporal dependencies and state uncertainty in traffic flow—while leveraging GPG for end-to-end policy optimization enabling probabilistic path inference from historical traffic sequences. Results: Experiments on the Sioux Falls network demonstrate that the method significantly improves on-time arrival probability, outperforming existing deterministic and time-varying models. It achieves both high decision accuracy and robustness in complex, stochastic road networks.

Technology Category

• Planning, Routing, and Scheduling: Planning with Language Models
• Reasoning under Uncertainty: Stochastic Optimization
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

With the rapidly increased number of vehicles in urban areas, existing road infrastructure struggles to accommodate modern traffic demands, resulting in the issue of congestion. This highlights the importance of efficient path planning strategies. However, most recent navigation models focus solely on deterministic or time-dependent networks, while overlooking the correlations and the stochastic nature of traffic flows. In this work, we address the reliable shortest path problem within stochastic transportation networks under certain dependencies. We propose a path planning solution that integrates the decision Transformer with the Generalized Policy Gradient (GPG) framework. Based on the decision Transformer's capability to model long-term dependencies, our proposed solution improves the accuracy and stability of path decisions. Experimental results on the Sioux Falls Network (SFN) demonstrate that our approach outperforms previous baselines in terms of on-time arrival probability, providing more accurate path planning solutions.

Problem

• Addresses reliable shortest path in stochastic transportation networks
• Integrates decision Transformer with Generalized Policy Gradient framework
• Improves accuracy and stability of path planning decisions

Innovation

• Generalized Policy Gradient with Decision Transformer integration
• Models long-term dependencies for accurate path planning
• Improves on-time arrival probability in stochastic networks