Event-Driven Reinforcement Learning Enables Long-Horizon Control in Semiconductor Fabrication

2026-06-09

AI Summary

This work addresses the complex scheduling and control challenges in semiconductor manufacturing arising from high dimensionality, stochasticity, stringent constraints, and long-horizon feedback. The authors propose a centralized deep reinforcement learning framework that models system dynamics as a discrete event-driven temporal process. They introduce a novel, general-purpose event-driven temporal difference method compatible with multiple policy optimization algorithms, supporting both offline and online training while offering strong scalability and transferability. Leveraging model-free deep reinforcement learning within a high-fidelity industrial simulation environment, the approach enables end-to-end control of large-scale production lines. Experimental results under diverse real-world operating conditions demonstrate significant improvements in throughput and equipment utilization, validating the method’s effectiveness and generalization capability.

Abstract

Reinforcement learning promises to optimize sequential decisions in large-scale systems. Semiconductor manufacturing systems are stochastic and highly constrained environments where heterogeneous wafers traverse hundreds of processing steps across extensive equipment networks. These characteristics yield complex, high-dimensional decision problems with delayed feedback and long-horizon requirements, complicating production planning and control. We propose a deep reinforcement learning framework for multi-objective policy optimization at this scale. Specifically, we formulate control as a centralized-agent problem, where a core policy coordinates system-wide decisions, while system evolution is represented as an interconnected temporal process driven by discrete events. Accordingly, we develop a tailored event-driven temporal-difference formulation that remains general and can be integrated with various policy optimization methods under relevant training settings. We investigate several core model-free algorithms incorporated into this framework and evaluate their effectiveness using high-fidelity simulations of diverse, industry-real operating scenarios. Across extensive validation experiments, agents trained in both offline and online settings show significant and consistent gains in throughput and utilization. We further evaluate performance and generalization across training phases, clarifying the relative strengths of alternative reinforcement learning formulations and algorithms. Overall, the results support the scalability, generality, and transferability of the proposed framework for controlling event-driven complex adaptive systems.

Problem

• semiconductor fabrication
• long-horizon control
• stochastic systems
• delayed feedback
• high-dimensional decision-making

Innovation

• event-driven reinforcement learning
• long-horizon control
• semiconductor fabrication
• temporal-difference learning
• centralized multi-objective policy