DreamQAS: Learning a Decision-Useful World Model for VQE-Efficient Quantum Architecture Search

2026-07-31

AI Summary

Traditional reinforcement learning suffers from inefficiency in quantum architecture search due to the repeated execution of costly variational quantum eigensolvers (VQEs). This work proposes a model-based reinforcement learning framework that constructs a world model oriented toward decision usefulness rather than precise energy prediction. By leveraging relative energy ranking, an uncertainty-aware pessimistic mechanism, and selective ground-truth verification, the method enables efficient exploration within the space of valid quantum circuits. It integrates recurrent stochastic prior ensembles, multi-step imagined rollouts, and a risk-aware coverage rejection mechanism. Experiments demonstrate that the approach achieves the lowest average frozen-policy energy error on four out of five molecular tasks and reduces real VQE calls by 1.6–10.6× under high-accuracy targets, significantly outperforming baselines in counterfactual action ranking utility and risk coverage.

Abstract

Reinforcement-learning-based quantum architecture search (RL-QAS) repeatedly optimizes a variational quantum eigensolver (VQE) after extending a circuit, although circuit construction and action legality are deterministic and known. We introduce DreamQAS, a model-based RL framework that preserves these exact circuit dynamics and learns only the expensive post-VQE feedback. A recurrent randomized-prior ensemble predicts an oracle-free score relative to an empirical energy frontier and supports multi-step imagined policy learning over explicit legal circuits. Ranking-based activation, uncertainty-aware pessimism and truncation, and selective real-VQE verification form a reliability-controlled learning loop. Under a common 15,000-episode budget and frozen evaluation for the RL methods, DreamQAS has the lowest mean frozen-policy energy error on four of five molecular tasks and the second-lowest on one. At fine-error targets reached by all seeds of both methods, it uses 1.6x to 2.0x fewer real VQE calls on four tasks and 10.6x fewer on BeH2-8q. Counterfactual action-ranking utility increases across all five tasks, with a mean increase of 0.346 and a 95 percent confidence interval of [0.185, 0.507], while direct greedy and beam use of the same model does not recover the gains of imagined policy learning. Ensemble disagreement also improves risk-coverage over random rejection on all three probed tasks. These results establish a world-model design for QAS whose value lies in decision-useful feedback rather than exact energy prediction.

Problem

• quantum architecture search
• variational quantum eigensolver
• world model
• reinforcement learning
• decision-useful feedback

Innovation

• model-based reinforcement learning
• quantum architecture search
• variational quantum eigensolver
• world model
• decision-useful feedback