Compositional Verification of Almost-Sure Büchi Objectives in MDPs

2025-08-18

AI Summary

This paper addresses the problem of verifying the existence of strategies that almost-surely satisfy Büchi objectives in Markov decision processes (MDPs) with known compositional structure. We propose a compositional verification framework based on string graphs, whose core innovation is the first formal definition of an “adequate exit set”—a necessary and sufficient statistic characterizing local strategy behavior—coupled with a recursive analysis mechanism integrating local reachability and exit-set computation. Based on this foundation, we design two algorithms: (i) a sound and complete bottom-up verification algorithm, and (ii) a polynomial-time iterative strategy refinement algorithm that avoids exponential complexity and enables efficient approximate verification. Experimental evaluation demonstrates that our approach significantly improves scalability and precision for Büchi property verification on large-scale compositional MDPs.

Technology Category

Planning, Routing, and Scheduling: Planning with Markov Models (MDPs, POMDPs)Reasoning under Uncertainty: Stochastic OptimizationMultiagent Systems: Mechanism Design

Application Category

Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphsEconomics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystemsResponsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

This paper studies the verification of almost-sure Büchi objectives in MDPs with a known, compositional structure based on string diagrams. In particular, we ask whether there is a strategy that ensures that a Büchi objective is almost-surely satisfied. We first show that proper exit sets -- the sets of exits that can be reached within a component without losing locally -- together with the reachability of a Büchi state are a sufficient and necessary statistic for the compositional verification of almost-sure Büchi objectives. The number of proper exit sets may grow exponentially in the number of exits. We define two algorithms: (1) A straightforward bottom-up algorithm that computes this statistic in a recursive manner to obtain the verification result of the entire string diagram and (2) a polynomial-time iterative algorithm which avoids computing all proper exit sets by performing iterative strategy refinement.

Problem

• Compositional verification of almost-sure Büchi objectives in MDPs
• Determining strategies for satisfying Büchi objectives almost-surely
• Avoiding exponential growth in proper exit set computations

Innovation

• Compositional structure using string diagrams
• Proper exit sets with Buechi state reachability
• Polynomial-time iterative strategy refinement algorithm