POMDPs for Autonomous Science Exploration

2026-08-04

AI Summary

This work addresses the challenge of integrating scientific reasoning into partially observable Markov decision process (POMDP) planning under high-dimensional observations, where existing information-theoretic approaches often compromise the rigorous uncertainty modeling inherent to POMDPs. To overcome this limitation, the paper introduces the Scientific Hypothesis Map POMDP (SHM-POMDP), which uniquely embeds a hierarchical probabilistic model into belief-space planning. This framework preserves full observational fidelity while performing decision branching at the level of inferred physical properties, thereby balancing navigation efficiency with scientific objectives. By integrating a learned observation model, a principled belief update mechanism, and an adaptive replanning strategy, the method achieves an 18.6% increase in reward and a 32.9% reduction in per-step computation time on an extended RockSample task, and yields 2.5 times the information gain of the best information-theoretic baseline in Cuprite hyperspectral geological exploration.

Abstract

Autonomous exploration missions require decision-making under sensor uncertainty and computational constraints, yet integrating scientific representations into POMDP planning has remained intractable due to high-dimensional observation spaces. Information-theoretic planners overcome this by assuming deterministic observations, sacrificing the principled uncertainty quantification that POMDPs provide. We introduce the Science Hypothesis Map POMDP (SHM-POMDP), which makes science-driven belief-space planning more tractable by branching on inferred physical properties rather than raw sensor data. This preserves full sensor information through learned observation models while enabling the planner to reason jointly about navigation and scientific properties under uncertainty. On an extended RockSample domain with 50-dimensional observations, SHM-POMDP achieves 18.6\% higher rewards and 32.9\% reduced computation time per step than continuous-observation baselines. On realistic geologic exploration using Cuprite hyperspectral data, SHM-POMDP achieves 2.5$\times$ higher information gain than the best information-theoretic baseline by maintaining beliefs and replanning adaptively---reaching 80\% of oracle performance using only uniform priors. These results demonstrate that integrating hierarchical probabilistic models into belief-space planning enables tractable, principled autonomous science that outperforms both traditional POMDP methods and science-aware information-theoretic approaches.

Problem

• POMDP
• autonomous science exploration
• high-dimensional observations
• belief-space planning
• sensor uncertainty

Innovation

• SHM-POMDP
• belief-space planning
• autonomous science exploration
• hierarchical probabilistic models
• information gain