ProcessLight: Process Supervision for Large Language Model Based Traffic Signal Control

2026-09-19

AI Summary

为解决大语言模型在交通信号控制中无法区分有效与错误推理步骤的问题，提出ProcessLight框架及STeP-PO方法，通过逐步信用分配优化推理过程。

Abstract

Large Language Models (LLMs) have recently been introduced into traffic signal control (TSC) as decision agents due to their strengths in human-readable reasoning generation. Yet, existing LLM TSC methods optimize only from final outcomes and fail to distinguish valid from flawed reasoning steps, causing useful or misleading steps to be jointly updated and thus impairing the model's learning of effective reasoning. To bridge this gap, we propose an LLM-based framework ProcessLight to decompose signal decisions into verifiable semantic steps. Building on ProcessLight, we further develop Step-wise Traffic Process Policy Optimization (STeP-PO), a novel reinforcement learning framework that optimizes structured reasoning processes through step-level credit assignment. Specifically, STeP-PO uses step quality scores to evaluate local reasoning quality and step importance to measure each step's influence on the final action, and then assigns step-level advantages over a semantic step tree structure. The resulting step-level advantages are propagated to reasoning tokens, enabling fine-grained policy optimization beyond outcome-only rewards. Extensive experiments over multiple real-world datasets demonstrate the superiority of our methods. Our code is available at https://github.com/wenzhaoabc/processlight.

Problem

• Large Language Models
• Traffic Signal Control
• Reasoning Steps
• Outcome Optimization
• Step-level Credit Assignment

Innovation

• ProcessLight
• Step-wise Traffic Process Policy Optimization
• step-level credit assignment
• semantic step tree structure
• fine-grained policy optimization