Policy-Based Deep Reinforcement Learning Hyperheuristics for Job-Shop Scheduling Problems

2026-01-16

AI Summary

This work proposes a policy-based deep reinforcement learning hyper-heuristic framework to address the challenge of dynamically selecting effective dispatching rules for the job shop scheduling problem (JSSP). The framework enables an agent to adaptively switch among low-level dispatching rules based on the current system state. It introduces two key innovations: an action pre-filtering mechanism to ensure unbiased evaluation of heuristics and a commitment mechanism to regulate the frequency of rule switching. By integrating both deterministic and stochastic policy selection strategies, the approach significantly outperforms conventional heuristics, metaheuristics, and existing neural network–based scheduling methods on standard JSSP benchmarks, demonstrating superior scheduling performance and training stability.

Technology Category

• Planning, Routing, and Scheduling: Learning for Planning and Scheduling
• Search and Optimization: Heuristic Search
• Humans and AI: Human-Aware Planning and Behavior Prediction

Application Category

• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

This paper proposes a policy-based deep reinforcement learning hyper-heuristic framework for solving the Job Shop Scheduling Problem. The hyper-heuristic agent learns to switch scheduling rules based on the system state dynamically. We extend the hyper-heuristic framework with two key mechanisms. First, action prefiltering restricts decision-making to feasible low-level actions, enabling low-level heuristics to be evaluated independently of environmental constraints and providing an unbiased assessment. Second, a commitment mechanism regulates the frequency of heuristic switching. We investigate the impact of different commitment strategies, from step-wise switching to full-episode commitment, on both training behavior and makespan. Additionally, we compare two action selection strategies at the policy level: deterministic greedy selection and stochastic sampling. Computational experiments on standard JSSP benchmarks demonstrate that the proposed approach outperforms traditional heuristics, metaheuristics, and recent neural network-based scheduling methods

Problem

• Job Shop Scheduling Problem
• Hyper-heuristics
• Dynamic Scheduling
• Makespan Optimization
• Scheduling Rules

Innovation

• policy-based deep reinforcement learning
• hyper-heuristic
• action prefiltering
• commitment mechanism
• job shop scheduling