Large Language Model Scaling Laws for Neural Quantum States in Quantum Chemistry

2025-09-16

AI Summary

This study investigates the scalability laws of neural quantum states (NQS) in quantum chemical tasks. Focusing on second-quantized molecular Hamiltonians, we construct trial wavefunctions using a Transformer architecture and optimize them via variational Monte Carlo under controlled computational resource constraints. We systematically analyze the interdependence among model size, training time, and accuracy—measured by absolute energy error and V-score. Crucially, we uncover the first empirical evidence of *nonlinear scaling* for NQS in quantum chemistry: the scaling behavior is highly sensitive to both the choice of loss function and the wavefunction parameterization scheme, thereby invalidating the linear scaling assumption common in large language models. Leveraging empirical data, we develop a generalizable performance prediction model, validate its scaling laws across diverse small molecules, and demonstrate high-fidelity ground-state simulations under optimal computational resource allocation.

Technology Category

• Machine Learning: Quantum Machine Learning
• Search and Optimization: Non-convex Optimization
• Natural Language Processing: Learning & Optimization for NLP

Application Category

• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Scaling laws have been used to describe how large language model (LLM) performance scales with model size, training data size, or amount of computational resources. Motivated by the fact that neural quantum states (NQS) has increasingly adopted LLM-based components, we seek to understand NQS scaling laws, thereby shedding light on the scalability and optimal performance--resource trade-offs of NQS ansatze. In particular, we identify scaling laws that predict the performance, as measured by absolute error and V-score, for transformer-based NQS as a function of problem size in second-quantized quantum chemistry applications. By performing analogous compute-constrained optimization of the obtained parametric curves, we find that the relationship between model size and training time is highly dependent on loss metric and ansatz, and does not follow the approximately linear relationship found for language models.

Problem

• Investigating scaling laws for neural quantum states in quantum chemistry
• Analyzing performance-resource trade-offs for transformer-based NQS ansatze
• Determining how model size and training time relate differently than LLMs

Innovation

• Transformer-based neural quantum states
• Scaling laws for quantum chemistry
• Compute-constrained optimization analysis