Optimal Constrained sc-LTL Planning in MDPs via Switching Policies

2026-08-05

AI Summary

This work addresses the challenge of synthesizing optimal policies in Markov decision processes (MDPs) that simultaneously satisfy a goal specification and safety constraints expressed in syntactically co-safe linear temporal logic (sc-LTL). The authors propose an efficient solution by transforming the original non-Markovian problem into a constrained reachability problem over an augmented MDP. They prove that optimality can be achieved within a policy class composed of finitely many static policies switched at appropriate times, enabling the reformulation of the synthesis problem as a tractable linear program. Experimental results in grid-world environments demonstrate that the approach guarantees safety while optimally achieving the specified objectives, offering both theoretical optimality and computational scalability.

Abstract

We study the synthesis of optimal policies for planning problems on Markov decision processes with both objectives and safety constraints specified in co-safe linear temporal logic (sc-LTL). Our problems are inherently non-Markovian due to the complexity of the sc-LTL specification and may require policy randomization to balance the objective and constraint. We propose a novel approach that reduces the constrained sc-LTL planning problem to a constrained reachability problem on an extended model. We then show that a class of switching policies constructed from stationary policies for the individual sc-LTL specifications is sufficient for optimality for the constrained reachability problem. Our finding enables a tractable linear program to compute the optimal policy. A grid world case study demonstrates that our switching policies can achieve the optimal trade-off between the objective and the safety constraint and validates both optimality and tractability.

Problem

• constrained planning
• Markov decision processes
• co-safe linear temporal logic
• optimal policy synthesis
• safety constraints

Innovation

• switching policies
• constrained sc-LTL planning
• Markov decision processes
• constrained reachability
• linear programming