Large-scale portfolio optimization with variational neural annealing

2025-07-09

AI Summary

We address large-scale portfolio optimization under realistic constraints—transaction costs and turnover limits—which formulate as computationally intractable mixed-integer nonlinear programs (MINLPs). To tackle this, we propose the first mapping of the problem onto an Ising-like Hamiltonian and introduce Variational Neural Annealing (VNA), a novel solver framework based on autoregressive neural networks. Our method integrates classical Ising modeling with dynamic finite-size scaling analysis, uncovering universal algorithmic behavior and polynomial annealing-time scaling across S&P 500, Russell 1000, and Russell 3000 indices. Experiments demonstrate that VNA achieves stable convergence on portfolios exceeding 2,000 assets, matching the solution quality of the commercial solver MOSEK while significantly outperforming it in convergence speed on high-difficulty instances. This work establishes the first application paradigm of variational neural annealing to financial portfolio optimization, bridging statistical physics-inspired computation with practical quantitative finance.

Technology Category

• Search and Optimization: Mixed Discrete/Continuous Search
• Constraint Satisfaction and Optimization: Mixed Discrete/Continuous Optimization
• Machine Learning: Optimization

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Economics, Online Markets and Human Computation: Social networks and social learning
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods

Abstract

Portfolio optimization is a routine asset management operation conducted in financial institutions around the world. However, under real-world constraints such as turnover limits and transaction costs, its formulation becomes a mixed-integer nonlinear program that current mixed-integer optimizers often struggle to solve. We propose mapping this problem onto a classical Ising-like Hamiltonian and solving it with Variational Neural Annealing (VNA), via its classical formulation implemented using autoregressive neural networks. We demonstrate that VNA can identify near-optimal solutions for portfolios comprising more than 2,000 assets and yields performance comparable to that of state-of-the-art optimizers, such as Mosek, while exhibiting faster convergence on hard instances. Finally, we present a dynamical finite-size scaling analysis applied to the S&P 500, Russell 1000, and Russell 3000 indices, revealing universal behavior and polynomial annealing time scaling of the VNA algorithm on portfolio optimization problems.

Problem

• Solving large-scale portfolio optimization under real-world constraints
• Mapping portfolio optimization to Ising-like Hamiltonian for efficient solution
• Demonstrating VNA's superior performance and scalability on complex portfolios

Innovation

• Uses Variational Neural Annealing for optimization
• Maps problem to Ising-like Hamiltonian model
• Employs autoregressive neural networks implementation