Technology Category
Planning, Routing, and Scheduling: Planning with Markov Models (MDPs, POMDPs)Reasoning under Uncertainty: Relational Probabilistic ModelsMachine Learning: Deep Generative Models & Autoencoders
Conditional Deep Generative Models for Belief State Planning
🤖 AI Summary
For high-dimensional continuous-state partially observable Markov decision processes (POMDPs), conventional particle filtering struggles to accurately represent posterior beliefs. This work introduces conditional deep generative models (cDGMs) for POMDP belief representation—the first such application—enabling flexible modeling of complex posterior distributions over high-dimensional continuous state-observation spaces and scalable sample generation. Our method is trained end-to-end on stochastic rollout trajectory data, learning the mapping from observation sequences to belief distributions in a data-driven manner. Evaluated on a large-scale continuous mineral exploration POMDP, the proposed approach significantly outperforms particle filtering baselines in both belief estimation accuracy and downstream planning performance. It effectively alleviates the expressivity bottleneck in belief representation and inference for high-dimensional POMDPs.
Technology Category
Application Category
📝 Abstract
Partially observable Markov decision processes (POMDPs) are used to model a wide range of applications, including robotics, autonomous vehicles, and subsurface problems. However, accurately representing the belief is difficult for POMDPs with high-dimensional states. In this paper, we propose a novel approach that uses conditional deep generative models (cDGMs) to represent the belief. Unlike traditional belief representations, cDGMs are well-suited for high-dimensional states and large numbers of observations, and they can generate an arbitrary number of samples from the posterior belief. We train the cDGMs on data produced by random rollout trajectories and show their effectiveness in solving a mineral exploration POMDP with a large and continuous state space. The cDGMs outperform particle filter baselines in both task-agnostic measures of belief accuracy as well as in planning performance.
Problem
Research questions and friction points this paper is trying to address.
Representing high-dimensional belief states in POMDPs
Handling large continuous state spaces in POMDPs
Improving belief accuracy and planning performance
Innovation
Methods, ideas, or system contributions that make the work stand out.
Uses conditional deep generative models (cDGMs)
Trains cDGMs on random rollout trajectories
Outperforms particle filter baselines in accuracy
🔎 Similar Papers
No similar papers found.
