Quantum-Brain: Quantum-Inspired Neural Network Approach to Vision-Brain Understanding

2024-11-20

arXiv.org

AI Summary

Existing vision–brain decoding models fail to explicitly model functional connectivity among brain regions, thereby limiting their capacity for semantic information extraction. To address this, we propose the first end-to-end quantum-inspired neural network that explicitly embeds inter-regional functional connectivity into a deep learning framework. Our approach introduces three novel components: a quantum-inspired voxel-wise modulation module, a phase-shift calibration module, and a measurement projection module—collectively enabling phase-sensitive, voxel-level connectivity modeling within a Hilbert space. By explicitly encoding functional connectivity—previously neglected in conventional models—our method significantly advances semantic alignment between visual stimuli and neural representations. On the Natural Scene Dataset, it achieves state-of-the-art performance: 95.1% top-1 accuracy in image retrieval from fMRI, 95.6% accuracy in brain-signal retrieval from images, and an Inception Score of 95.3 for fMRI-to-image reconstruction.

Technology Category

• Computer Vision: Multi-modal Vision
• Cognitive Modeling & Cognitive Systems: Neural Spike Coding
• Machine Learning: Quantum Machine Learning

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web query analysis, representation and understanding
• Graph Algorithms and Modeling for the Web: Querying, indexing, and retrieval in Web-related graphs

Abstract

Vision-brain understanding aims to extract semantic information about brain signals from human perceptions. Existing deep learning methods for vision-brain understanding are usually introduced in a traditional learning paradigm missing the ability to learn the connectivities between brain regions. Meanwhile, the quantum computing theory offers a new paradigm for designing deep learning models. Motivated by the connectivities in the brain signals and the entanglement properties in quantum computing, we propose a novel Quantum-Brain approach, a quantum-inspired neural network, to tackle the vision-brain understanding problem. To compute the connectivity between areas in brain signals, we introduce a new Quantum-Inspired Voxel-Controlling module to learn the impact of a brain voxel on others represented in the Hilbert space. To effectively learn connectivity, a novel Phase-Shifting module is presented to calibrate the value of the brain signals. Finally, we introduce a new Measurement-like Projection module to present the connectivity information from the Hilbert space into the feature space. The proposed approach can learn to find the connectivities between fMRI voxels and enhance the semantic information obtained from human perceptions. Our experimental results on the Natural Scene Dataset benchmarks illustrate the effectiveness of the proposed method with Top-1 accuracies of 95.1% and 95.6% on image and brain retrieval tasks and an Inception score of 95.3% on fMRI-to-image reconstruction task. Our proposed quantum-inspired network brings a potential paradigm to solving the vision-brain problems via the quantum computing theory.

Problem

• Extract semantic info from brain signals via perceptions
• Learn brain region connectivities missing in current methods
• Enhance vision-brain understanding using quantum-inspired models

Innovation

• Quantum-Inspired Voxel-Controlling module for brain connectivity
• Phase-Shifting module to calibrate brain signals
• Measurement-like Projection module for feature space