Progress-SQL: Improving Reinforcement Learning for Text-to-SQL via Progressive Rewards

2026-06-04

AI Summary

This work proposes Progress-SQL, a novel framework that addresses the limitations of existing reinforcement learning–based Text-to-SQL methods, which rely on a single reward signal and struggle to effectively guide iterative SQL refinement. Progress-SQL introduces, for the first time, an Oracle-guided Diagnostic Tree (ODT) to abstract SQL queries at the clause level, enabling structured analysis. By integrating both structural and lexical alignment between predicted and ground-truth queries, the framework generates progressive, fine-grained rewards that provide precise feedback during interactive, multi-turn SQL generation. Empirical evaluations demonstrate that Progress-SQL achieves significant performance gains on benchmark datasets including BIRD and Spider, as well as their robustness-oriented variants, while also enhancing model resilience to perturbed inputs.

Abstract

Reinforcement learning has recently shown promise in improving large language models for Text-to-SQL generation, yet existing methods typically optimize one-shot rewards defined over a single SQL state. Such rewards provide limited guidance for iterative SQL correction and are insufficient to capture the improvement of multi-turn SQL refinement. In this paper, we propose Progress-SQL, a multi-turn reinforcement learning framework with progressive rewards for Text-to-SQL. Our approach introduces an Oracle-guided Diagnostic Tree (ODT), which abstracts SQL queries into clause-level structural profiles and produces diagnostic feedback for next-turn refinement. To provide dense and robust reward signals, we combine ODT-based structural alignment with lexical alignment and define a progressive reward that measures the improvement from the initial SQL to the final SQL. We further incorporate a progression latency reward that favors earlier correctness and an execution status reward that encourages recovery from the invalid SQL. Experiments on BIRD, Spider, and Spider robustness variants demonstrate that our method consistently improves Text-to-SQL performance across both primary and robustness evaluations.

Problem

• Text-to-SQL
• reinforcement learning
• progressive rewards
• multi-turn refinement
• SQL correction

Innovation

• Progressive Rewards
• Oracle-guided Diagnostic Tree
• Multi-turn Reinforcement Learning
• Text-to-SQL
• Structural Alignment