Decision Transformer for Enhancing Neural Local Search on the Job Shop Scheduling Problem

2024-09-04

arXiv.org

AI Summary

For the Job-Shop Scheduling Problem (JSSP), this paper introduces the first integration of Decision Transformers (DTs) into the Neural Local Search (NLS) framework, aiming to improve the trade-off between solution quality and computational efficiency for deep reinforcement learning agents. Unlike conventional NLS approaches that rely on implicit policy learning, our method explicitly models and optimizes search trajectories via sequence modeling and trajectory distillation, thereby enhancing per-step decision quality. Experiments on standard JSSP benchmarks demonstrate state-of-the-art performance among machine-learning-augmented search methods: our approach significantly outperforms existing NLS and RL baselines in solution quality, with particularly pronounced gains under medium-to-long inference-time budgets. These results validate the effectiveness and scalability of DT-driven trajectory optimization for combinatorial optimization within the local search paradigm.

Technology Category

• Search and Optimization: Learning to Search
• Planning, Routing, and Scheduling: Learning for Planning and Scheduling
• Natural Language Processing: Learning & Optimization for NLP

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs

Abstract

The job shop scheduling problem (JSSP) and its solution algorithms have been of enduring interest in both academia and industry for decades. In recent years, machine learning (ML) is playing an increasingly important role in advancing existing and building new heuristic solutions for the JSSP, aiming to find better solutions in shorter computation times. In this paper we build on top of a state-of-the-art deep reinforcement learning (DRL) agent, called Neural Local Search (NLS), which can efficiently and effectively control a large local neighborhood search on the JSSP. In particular, we develop a method for training the decision transformer (DT) algorithm on search trajectories taken by a trained NLS agent to further improve upon the learned decision-making sequences. Our experiments show that the DT successfully learns local search strategies that are different and, in many cases, more effective than those of the NLS agent itself. In terms of the tradeoff between solution quality and acceptable computational time needed for the search, the DT is particularly superior in application scenarios where longer computational times are acceptable. In this case, it makes up for the longer inference times required per search step, which are caused by the larger neural network architecture, through better quality decisions per step. Thereby, the DT achieves state-of-the-art results for solving the JSSP with ML-enhanced search.

Problem

• Enhances Neural Local Search
• Improves Job Shop Scheduling
• Optimizes decision-making sequences

Innovation

• Decision Transformer enhances Neural Local Search
• DT learns effective local search strategies
• DT achieves state-of-the-art JSSP results