Improved Quantum Algorithms for Reinforcement Learning Under a Generative Model

2026-08-03

AI Summary

This work addresses the problem of efficiently computing near-optimal policies for both finite-horizon and infinite-horizon discounted Markov decision processes (MDPs) under a generative model. We propose a novel quantum reinforcement learning algorithm that, for the first time, deeply integrates sample-optimal classical RL techniques with quantum subroutines—specifically quantum mean estimation, quantum maximum finding, and value iteration. The resulting approach achieves significantly reduced query complexity, outperforming existing methods on both MDP settings while nearly matching known quantum query lower bounds. This advance demonstrates dual innovation in both theoretical foundations and practical algorithmic design.

Abstract

Reinforcement learning is a subfield of machine learning that studies how an agent interacts with an environment in order to extract as large a reward as possible. A standard approach to study such interaction is through Markov Decision Processes (MDPs) and the task of choosing an optimal policy --- a function that tells the agent which action to take. In this work, we study two types of MDPs --- finite-horizon and infinite-horizon discounted --- and propose new quantum algorithms for computing approximate optimal policies. Our quantum algorithms are based on a new combination of standard value iteration and quantum subroutines like quantum mean estimation and quantum maximum finding, overall enhanced with techniques from sample-optimal classical algorithms. Our resulting query complexities improve upon previous works, thus approaching already established quantum lower bounds.

Problem

• Reinforcement Learning
• Markov Decision Processes
• Quantum Algorithms
• Optimal Policy
• Generative Model

Innovation

• quantum reinforcement learning
• Markov Decision Processes
• quantum mean estimation
• quantum maximum finding
• value iteration