Automata-Theoretic Verification of Interval Markov Decision Processes

2026-09-18

AI Summary

本文解决了区间马尔可夫决策过程(IMDPs)的验证问题，针对不确定性过渡概率，通过自动机理论方法，并区分稳定与不稳定IMDPs，提出相应验证算法。

Abstract

Interval Markov decision processes (IMDPs) provide a natural framework for modeling stochastic systems with uncertain transition probabilities, represented by probability intervals and resolved adversarially. Such uncertainty arises naturally, for example, when the transition model is learned from finite data or obtained through model-based reinforcement learning. In this paper, we study the automata-theoretic verification of IMDPs against rich temporal specifications, including all LTL specifications, by considering the broader class of ω-regular objectives. We show that classical automata-theoretic verification techniques extend to IMDPs, but with a sharp distinction determined by the structure of the transition intervals. For stable IMDPs, where either the upper bound is zero or the lower bound is strictly positive, verification reduces to ordinary MDP analysis and can be carried out using the standard automata used in that setting (good-for-MDP automata). For unstable IMDPs, where intervals may include zero while the upper bound is strictly positive, verification becomes game-like and requires automata whose nondeterminism can be resolved on the fly (good-for-games automata). Building on these insights, we develop algorithms for verifying ω-regular specifications over IMDPs and derive probabilistic guarantees when the interval model is learned from sampled data. The resulting framework enables principled verification of stochastic systems under probabilistic model uncertainty, connecting automata-based verification with data-driven stochastic modeling.

Problem

• Interval Markov Decision Processes
• uncertain transition probabilities
• temporal specifications
• automata-theoretic verification
• \\omega-regular objectives

Innovation

• Interval Markov Decision Processes
• Automata-Theoretic Verification
• \\omega-regular Specifications
• Good-for-Games Automata
• Probabilistic Guarantees