Staged Hybridisation for Visual Quantum Reinforcement Learning via Knowledge Distillation

2026-06-29

AI Summary

This work addresses the challenges of end-to-end training quantum reinforcement learning policies under high-dimensional visual inputs, which are hindered by the limited expressivity and optimization instability of variational quantum circuits (VQCs). To circumvent these issues, the authors propose a staged knowledge distillation framework: first, a classical vision-based teacher model is trained and its encoder is frozen to serve as a fixed feature extractor; subsequently, the teacher’s policy behavior is distilled into a lightweight downstream head—either classical or quantum—effectively reframing visual quantum reinforcement learning as a compact policy-head learning problem. Empirical validation on CartPole Pixels and Acrobot Pixels demonstrates that shallow VQC policy heads can achieve non-trivial control performance approaching that of the teacher. Angle encoding proves robust and efficient, whereas amplitude encoding, though maximally compact, exhibits greater sensitivity.

Abstract

Visual environments are a demanding setting for quantum reinforcement learning (QRL): high-dimensional observations, unstable RL optimisation, and constrained variational quantum circuits (VQCs) are difficult to train jointly. This paper studies knowledge distillation (KD) as a staged hybridisation strategy for visual QRL. Instead of training a hybrid visual agent end-to-end from pixels, we first train a classical visual teacher, freeze its encoder as a feature interface, and distil the teacher's policy behaviour into compact downstream heads. These heads can be classical or VQC-based, enabling small quantum-compatible students to be evaluated under the same frozen representation as compact classical controls. We evaluate the pipeline on CartPole Pixels and Acrobot Pixels. The results show that staged KD enables shallow VQC heads to acquire non-trivial visual-control behaviour in settings where direct pixel-based training would be substantially more difficult. Angle-encoded VQC heads retain near-teacher performance, while amplitude-encoded heads push compactness to an extreme regime, at the cost of greater fragility, stronger budget sensitivity, and higher simulation time. Overall, staged KD reframes visual QRL as a compact-head learning problem, opening a practical route for training small quantum-compatible policies outside the standard end-to-end RL loop.

Problem

• visual quantum reinforcement learning
• high-dimensional observations
• unstable RL optimisation
• constrained variational quantum circuits
• quantum-compatible policies

Innovation

• staged hybridisation
• knowledge distillation
• quantum reinforcement learning
• variational quantum circuits
• visual control