Max-Policy Iteration, Revisited

2026-06-09

AI Summary

This work addresses the limitations of traditional max-policy iteration, which relies on costly mathematical optimization and widening operators. The authors propose a novel policy iteration framework based solely on value iteration: standard value iteration replaces the need for solving systems of integer and floating-point equations via optimization, while min-policy iteration substitutes linear programming to handle max-min affine equation systems over the rationals. The study establishes, for the first time, that pure value iteration can guarantee termination of max-policy iteration. The proposed max-min policy iteration converges to the least solution for bounded systems, extends naturally to unbounded cases, and generates certificates of correctness and optimality suitable for formal verification. The method achieves precise and convergent program invariant analysis across multiple numerical domains.

Abstract

Max-policy iteration is an approach to computing precise numeric program invariants by successive attempts at resolving maximum operators and reduction to mathematical optimization. Mathematical optimization, though, may be expensive. Here, we show, for max-policy iteration on systems of equations over integers as well as over floating point numbers, that mathematical optimization can be replaced by plain value iteration -- which is still guaranteed to terminate. As an application, a precise bound analysis for integer or floating point variables is obtained, avoiding widening operators altogether. We also consider max-policy iteration over the rational numbers, where the right-hand sides are maxima of minima of affine combinations of unknowns. We propose min-policy iteration as an alternative to linear programming for solving the optimization problems posed by max-policy iteration. We prove that max-min policy iteration is guaranteed to return the least solution for bounded systems. We also show how to extend this algorithm to unbounded systems, and how to construct certificates of soundness as well as of optimality of the computed results.

Problem

• max-policy iteration
• program invariants
• value iteration
• bound analysis
• mathematical optimization

Innovation

• max-policy iteration
• value iteration
• min-policy iteration
• program invariants
• optimization-free analysis