Deep Learning Algorithms for Mean Field Optimal Stopping in Finite Space and Discrete Time

2024-10-11

arXiv.org

AI Summary

Multi-agent optimal stopping (MAOS) becomes computationally intractable in large-scale settings due to exponential growth in state and policy spaces. Method: This paper introduces the mean-field optimal stopping (MFOS) framework—first systematically formulated for finite-state, discrete-time systems—and establishes its uniform approximation guarantee to MAOS under mild regularity conditions. Leveraging mean-field control theory, we derive a dynamic programming principle for MFOS and propose two deep learning algorithms: full-trajectory simulation learning and backward-induction training, both scalable to 300-dimensional state spaces. Results: Evaluated across six benchmark problems, the proposed approach achieves significant improvements in computational scalability and policy accuracy over existing methods, providing a theoretically grounded and practically scalable foundation for high-dimensional multi-agent sequential decision-making.

Technology Category

• Multiagent Systems: Multiagent Learning
• Search and Optimization: Learning to Search
• Reasoning under Uncertainty: Sequential Decision Making

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Optimal stopping is a fundamental problem in optimization that has found applications in risk management, finance, economics, and recently in the fields of computer science. We extend the standard framework to a multi-agent setting, named multi-agent optimal stopping (MAOS), where a group of agents cooperatively solves finite-space, discrete-time optimal stopping problems. Solving the finite-agent case is computationally prohibitive when the number of agents is very large, so this work studies the mean field optimal stopping (MFOS) problem, obtained as the number of agents approaches infinity. We prove that MFOS provides a good approximate solution to MAOS. We also prove a dynamic programming principle (DPP), based on the theory of mean field control. We then propose two deep learning methods: one simulates full trajectories to learn optimal decisions, whereas the other leverages DPP with backward induction; both methods train neural networks for the optimal stopping decisions. We demonstrate the effectiveness of these approaches through numerical experiments on 6 different problems in spatial dimension up to 300. To the best of our knowledge, this is the first work to study MFOS in finite space and discrete time, and to propose efficient and scalable computational methods for this type of problem.

Problem

• Extends optimal stopping to multi-agent cooperative settings
• Solves computational challenges in large-scale MAOS via mean-field approximation
• Proposes deep learning methods for scalable optimal stopping policies

Innovation

• Extends optimal stopping to multi-agent setting
• Uses deep learning for mean-field approximation
• Proposes two neural network training methods