Transformer-based Stagewise Decomposition for Large-Scale Multistage Stochastic Optimization

2024-04-03

International Conference on Machine Learning

AI Summary

Stochastic Dual Dynamic Programming (SDDP) and other stage-wise decomposition algorithms for large-scale multistage stochastic programming (MSP) suffer from rapidly increasing computational complexity due to the accumulation of cutting planes over stages. Method: This paper pioneers the integration of the Transformer architecture into stochastic dynamic programming, proposing a sequence modeling–based value function learning framework. It takes subgradient cutting planes as input and employs a Transformer encoder to perform end-to-end, learnable, piecewise-linear approximation of time-series value functions—replacing hand-crafted cut generation. Contribution/Results: Evaluated on standard benchmark instances, the method achieves significant reductions in solution time while preserving solution quality. It demonstrates an exceptional trade-off between accuracy and efficiency, offering a scalable, data-driven paradigm for large-scale stochastic optimization.

Technology Category

• Reasoning under Uncertainty: Stochastic Optimization
• Search and Optimization: Learning to Search
• Planning, Routing, and Scheduling: Planning with Markov Models (MDPs, POMDPs)

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Solving large-scale multistage stochastic programming (MSP) problems poses a significant challenge as commonly used stagewise decomposition algorithms, including stochastic dual dynamic programming (SDDP), face growing time complexity as the subproblem size and problem count increase. Traditional approaches approximate the value functions as piecewise linear convex functions by incrementally accumulating subgradient cutting planes from the primal and dual solutions of stagewise subproblems. Recognizing these limitations, we introduce TranSDDP, a novel Transformer-based stagewise decomposition algorithm. This innovative approach leverages the structural advantages of the Transformer model, implementing a sequential method for integrating subgradient cutting planes to approximate the value function. Through our numerical experiments, we affirm TranSDDP's effectiveness in addressing MSP problems. It efficiently generates a piecewise linear approximation for the value function, significantly reducing computation time while preserving solution quality, thus marking a promising progression in the treatment of large-scale multistage stochastic programming problems.

Problem

• Address large-scale multistage stochastic optimization challenges
• Reduce computation time for stochastic dual dynamic programming
• Improve value function approximation using Transformer models

Innovation

• Transformer-based stagewise decomposition
• Sequential subgradient cutting plane integration
• Efficient piecewise linear value function approximation