Benign Overfitting with Quantum Kernels

2025-03-21

AI Summary

Quantum kernel methods suffer from kernel matrix degeneracy—manifesting as near-identity behavior—poor generalization, and overfitting, primarily due to exponential concentration of quantum states in high-dimensional Hilbert spaces. To address this, we propose a local-global quantum kernel framework: building upon quantum circuit embeddings, it jointly leverages subsystem-local projective measurements and full-system fidelity computations to construct a novel kernel function that balances discriminability and robustness. Crucially, we introduce the classical “benign overfitting” theory into quantum kernel design for the first time, thereby overcoming the generalization bottlenecks inherent in conventional fidelity-based kernels. Experiments demonstrate that our kernel significantly alleviates kernel matrix ill-conditioning, improves SVM classification accuracy, and maintains strong generalization performance—even in high-dimensional and overparameterized regimes.

Technology Category

• Machine Learning: Kernel Methods
• Search and Optimization: Learning to Search
• Computer Vision: Adversarial Attacks & Robustness

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Quantum kernels quantify similarity between data points by measuring the inner product between quantum states, computed through quantum circuit measurements. By embedding data into quantum systems, quantum kernel feature maps, that may be classically intractable to compute, could efficiently exploit high-dimensional Hilbert spaces to capture complex patterns. However, designing effective quantum feature maps remains a major challenge. Many quantum kernels, such as the fidelity kernel, suffer from exponential concentration, leading to near-identity kernel matrices that fail to capture meaningful data correlations and lead to overfitting and poor generalization. In this paper, we propose a novel strategy for constructing quantum kernels that achieve good generalization performance, drawing inspiration from benign overfitting in classical machine learning. Our approach introduces the concept of local-global quantum kernels, which combine two complementary components: a local quantum kernel based on measurements of small subsystems and a global quantum kernel derived from full-system measurements. Through numerical experiments, we demonstrate that local-global quantum kernels exhibit benign overfitting, supporting the effectiveness of our approach in enhancing quantum kernel methods.

Problem

• Designing effective quantum feature maps for kernels
• Preventing exponential concentration in quantum kernel matrices
• Achieving good generalization in quantum kernel methods

Innovation

• Local-global quantum kernels combine subsystems
• Quantum kernels exploit high-dimensional Hilbert spaces
• Benign overfitting inspired strategy enhances generalization