Enhancing Automated Program Repair via Faulty Token Localization and Quality-Aware Patch Refinement

2025-11-22

AI Summary

Existing LLM-based automated program repair (APR) methods rely on coarse-grained test feedback and lack fine-grained error attribution, resulting in low repair efficiency and severe error propagation. To address this, we propose TokenRepair—the first dual-layer APR framework integrating internal reflection and external feedback. It first identifies potentially erroneous tokens via context-aware, token-level uncertainty analysis; then performs precise patch generation through chain-of-thought-guided local rewriting and quality-aware iterative optimization—avoiding global regeneration to significantly improve repair stability and accuracy. Evaluated on Defects4J 1.2 and HumanEval-Java, TokenRepair successfully repairs 88 and 139 bugs, respectively, outperforming state-of-the-art methods by 8.2%–34.9%. These results empirically validate the effectiveness and generalizability of our token-level fine-grained reflection mechanism.

Technology Category

• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Planning, Routing, and Scheduling: Replanning and Plan Repair
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Search and Retrieval-Augmented AI: Large language models for search
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Large language models (LLMs) have recently demonstrated strong potential for automated program repair (APR). However, existing LLM-based techniques primarily rely on coarse-grained external feedback (e.g.,test results) to guide iterative patch generation, while lacking fine-grained internal signals that reveal why a patch fails or which parts of the generated code are likely incorrect. This limitation often leads to inefficient refinement, error propagation, and suboptimal repair performance. In this work, we propose TokenRepair, a novel two-level refinement framework that enhances APR by integrating internal reflection for localizing potentially faulty tokens with external feedback for quality-aware patch refinement. Specifically, TokenRepair first performs internal reflection by analyzing context-aware token-level uncertainty fluctuations to identify suspicious or low-confidence tokens within a patch. It then applies Chain-of-Thought guided rewriting to refine only these localized tokens, enabling targeted and fine-grained correction. To further stabilize the iterative repair loop, TokenRepair incorporates a quality-aware external feedback mechanism that evaluates patch quality and filters out low-quality candidates before refinement. Experimental results show that TokenRepair achieves new state-of-the-art repair performance, correctly fixing 88 bugs on Defects4J 1.2 and 139 bugs on HumanEval-Java, demonstrating substantial improvements ranging from 8.2% to 34.9% across all models on Defects4J 1.2 and from 3.3% to 16.1% on HumanEval-Java.

Problem

• Improving automated program repair by localizing faulty tokens and refining patches
• Addressing inefficient patch refinement and error propagation in LLM-based techniques
• Enhancing repair performance through fine-grained internal signals and quality feedback

Innovation

• Localizes faulty tokens using internal uncertainty analysis
• Refines patches via Chain-of-Thought guided token rewriting
• Filters low-quality patches with quality-aware external feedback