Deep RL Dual Sourcing Inventory Management with Supply and Capacity Risk Awareness

2025-07-18

AI Summary

This paper addresses multi-source, multi-period inventory management in large-scale stochastic supply chain optimization, explicitly accounting for supply uncertainty and capacity constraints. We propose a modular deep reinforcement learning framework: physical constraints are decoupled into composable, pre-trained deep learning modules that separately model stochastic demand and supply processes; a constraint coordination mechanism is designed to jointly predict dual costs—inventory holding and stockout—under cross-product resource competition, thereby circumventing optimization pitfalls inherent in end-to-end modeling. The approach significantly improves computational efficiency and policy robustness, outperforming baseline methods on large-scale real-world datasets. Our core contribution lies in introducing an interpretable, reusable modular paradigm that enables efficient, scalable sequential decision-making under complex stochastic constraints.

Technology Category

Reasoning under Uncertainty: Stochastic OptimizationConstraint Satisfaction and Optimization: Constraint Learning and AcquisitionSearch and Optimization: Learning to Search

Application Category

Economics, Online Markets and Human Computation: Cost models of using LLMs in production systemsGraph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphsSearch and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

In this work, we study how to efficiently apply reinforcement learning (RL) for solving large-scale stochastic optimization problems by leveraging intervention models. The key of the proposed methodology is to better explore the solution space by simulating and composing the stochastic processes using pre-trained deep learning (DL) models. We demonstrate our approach on a challenging real-world application, the multi-sourcing multi-period inventory management problem in supply chain optimization. In particular, we employ deep RL models for learning and forecasting the stochastic supply chain processes under a range of assumptions. Moreover, we also introduce a constraint coordination mechanism, designed to forecast dual costs given the cross-products constraints in the inventory network. We highlight that instead of directly modeling the complex physical constraints into the RL optimization problem and solving the stochastic problem as a whole, our approach breaks down those supply chain processes into scalable and composable DL modules, leading to improved performance on large real-world datasets. We also outline open problems for future research to further investigate the efficacy of such models.

Problem

• Optimizing large-scale stochastic inventory management with RL
• Forecasting supply chain risks using deep learning models
• Handling cross-product constraints via modular DL approaches

Innovation

• Deep RL models for stochastic supply chain forecasting
• Constraint coordination mechanism for dual costs
• Composable DL modules for scalable solutions