Is attention all you need to solve the correlated electron problem?

2025-02-07

AI Summary

This work addresses the computational challenge of determining ground states in strongly correlated electronic systems—particularly moiré quantum materials. We propose the first variational many-body wavefunction ansatz based on self-attention mechanisms. The ansatz models the wavefunction as a permutation-equivariant function of electron coordinates, leveraging self-attention to efficiently capture long-range quantum correlations. Crucially, the number of variational parameters scales quadratically with the electron count (N) (i.e., (O(N^2))), circumventing the exponential scaling inherent in conventional approaches. Integrated within a variational Monte Carlo framework, the method achieves high-accuracy, unbiased calculations of ground-state energies and physical observables for systems including twisted bilayer graphene. Our core contribution is the first incorporation of self-attention into quantum many-body wavefunction construction, markedly enhancing both feasibility and efficiency for large-scale simulations of strongly correlated solids.

Technology Category

• Machine Learning: Quantum Machine Learning
• Computer Vision: Large Vision Models
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Economics, Online Markets and Human Computation: Architectures and workflows that use LLMs for crowd work
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

The attention mechanism has transformed artificial intelligence research by its ability to learn relations between objects. In this work, we explore how a many-body wavefunction ansatz constructed from a large-parameter self-attention neural network can be used to solve the interacting electron problem in solids. By a systematic neural-network variational Monte Carlo study on a moir'e quantum material, we demonstrate that the self-attention ansatz provides an accurate, efficient, and unbiased solution. Moreover, our numerical study finds that the required number of variational parameters scales roughly as $N^2$ with the number of electrons, which opens a path towards efficient large-scale simulations.

Problem

• Exploring attention for electron problem
• Self-attention neural network for solids
• Efficient large-scale simulation parameters

Innovation

• Self-attention neural network
• Neural-network variational Monte Carlo
• Efficient large-scale simulations