Quantum Reinforcement Learning with Dynamic-Circuit Qubit Reuse and Grover-Based Trajectory Optimization

2025-09-19

AI Summary

To address the challenge of balancing limited qubit resources and trajectory fidelity on Noisy Intermediate-Scale Quantum (NISQ) devices, this work proposes a fully quantum reinforcement learning (RL) framework grounded in the quantum Markov decision process (QMDP), enabling end-to-end quantum encoding of states, actions, and rewards. We introduce a dynamic-circuit-based qubit reuse mechanism—reducing physical qubit count from 7T to 7—combined with mid-circuit measurement-and-reset and dedicated quantum arithmetic modules for efficient trajectory return computation. Furthermore, we integrate Grover’s search to amplify amplitudes of high-return trajectories, accelerating convergence to optimal policies. Simulation results demonstrate a 66% reduction in qubit consumption while preserving trajectory fidelity. Crucially, the framework is experimentally validated on the IBM Heron processor, marking the first scalable deployment of fully quantum RL on real, noisy, intermediate-scale quantum hardware.

Technology Category

• Machine Learning: Quantum Machine Learning
• Search and Optimization: Learning to Search
• Intelligent Robots: Learning & Optimization for ROB

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

A fully quantum reinforcement learning framework is developed that integrates a quantum Markov decision process, dynamic circuit-based qubit reuse, and Grover's algorithm for trajectory optimization. The framework encodes states, actions, rewards, and transitions entirely within the quantum domain, enabling parallel exploration of state-action sequences through superposition and eliminating classical subroutines. Dynamic circuit operations, including mid-circuit measurement and reset, allow reuse of the same physical qubits across multiple agent-environment interactions, reducing qubit requirements from 7*T to 7 for T time steps while preserving logical continuity. Quantum arithmetic computes trajectory returns, and Grover's search is applied to the superposition of these evaluated trajectories to amplify the probability of measuring those with the highest return, thereby accelerating the identification of the optimal policy. Simulations demonstrate that the dynamic-circuit-based implementation preserves trajectory fidelity while reducing qubit usage by 66 percent relative to the static design. Experimental deployment on IBM Heron-class quantum hardware confirms that the framework operates within the constraints of current quantum processors and validates the feasibility of fully quantum multi-step reinforcement learning under noisy intermediate-scale quantum conditions. This framework advances the scalability and practical application of quantum reinforcement learning for large-scale sequential decision-making tasks.

Problem

• Develops quantum reinforcement learning with qubit reuse
• Optimizes trajectories using Grover's algorithm quantum search
• Reduces qubit requirements for multi-step decision tasks

Innovation

• Dynamic circuit qubit reuse reduces hardware requirements
• Grover's algorithm optimizes quantum trajectory search
• Fully quantum framework eliminates classical subroutines