debugging strategies

Systematic methods for reproducing, isolating, and fixing faults in prompts, code, or visualizations, including iterative refinement, failure analysis, triage, and deciding corrective actions or rollbacks based on test outcomes.

debuggingstrategies

12-Month Skill Trend

Momentum and market value over time
Trending
Score
+20 in 12 mo
96
12 mo agoNow
Career
Value
+$12K in 12 mo
$42K/year
12 mo agoNow

Recommended Survey Paper

Quick overview of the field
View more

Must-Read Papers

Most classic and influential ideas
View more

In industrial quality inspection, anomaly detection suffers from poor robustness due to high noise levels and sparse defective samples. To address this, we propose Iterative Refinement of Pseudo-labels (IRP), a self-supervised method that alternately evaluates sample credibility and removes misleading instances under feature-space consistency constraints—effectively purifying the training set dynamically without human annotations and generating high-fidelity self-supervised signals. IRP introduces the novel paradigm of “iterative data refinement,” significantly enhancing model robustness against label noise and cross-domain generalization capability. Evaluated on KSDD2 and MVTec AD benchmarks, IRP consistently outperforms existing unsupervised and self-supervised methods. Notably, under high-noise conditions, it achieves substantial improvements in detection accuracy and reduces false positive rates by over 25%.

Enhances defect detection accuracy in industrial quality control.Improves model performance by removing misleading data points.Outperforms traditional models in noisy industrial environments.

Enhancing Automated Program Repair via Faulty Token Localization and Quality-Aware Patch Refinement

Nov 22, 2025
JK
Jiaolong Kong
🏛️ Singapore Management University

Existing LLM-based automated program repair (APR) methods rely on coarse-grained test feedback and lack fine-grained error attribution, resulting in low repair efficiency and severe error propagation. To address this, we propose TokenRepair—the first dual-layer APR framework integrating internal reflection and external feedback. It first identifies potentially erroneous tokens via context-aware, token-level uncertainty analysis; then performs precise patch generation through chain-of-thought-guided local rewriting and quality-aware iterative optimization—avoiding global regeneration to significantly improve repair stability and accuracy. Evaluated on Defects4J 1.2 and HumanEval-Java, TokenRepair successfully repairs 88 and 139 bugs, respectively, outperforming state-of-the-art methods by 8.2%–34.9%. These results empirically validate the effectiveness and generalizability of our token-level fine-grained reflection mechanism.

Addressing inefficient patch refinement and error propagation in LLM-based techniquesEnhancing repair performance through fine-grained internal signals and quality feedbackImproving automated program repair by localizing faulty tokens and refining patches

Automated Repair of C Programs Using Large Language Models

Sep 02, 2025
MF
Mahdi Farzandway
🏛️ University of Tehran

This paper addresses the challenge of automated repair of C programs by proposing the first closed-loop framework that integrates spectrum-based fault localization (SBFL) with large language model (LLM)-driven chain-of-thought (CoT) reasoning. Methodologically, the approach employs runtime test feedback to drive iterative refinement: SBFL initially identifies suspicious statements; an LLM generates patches using structured CoT prompting; and execution feedback guides successive improvements until all tests pass. The key contribution lies in the explicit coupling of statistical program analysis with symbolic LLM reasoning—enhancing fault attribution accuracy and mitigating recurrent patch failures. Evaluated on the Codeflaws benchmark (3,902 defects), the method achieves a repair accuracy of 44.93%, outperforming the GPT-4+CoT baseline by 3.61%, thereby validating the efficacy of analysis-guided iterative reasoning for program repair.

Automating C program repair using Large Language ModelsImproving patch accuracy through iterative feedback and structured promptingIntegrating statistical analysis with LLM reasoning for debugging

LLMShot: Reducing snapshot testing maintenance via LLMs

Jul 14, 2025
EB
Ergün Batuhan Kaynak
🏛️ Bilkent University

UI snapshot testing suffers from high false-positive rates due to frequent interface changes, and manual differentiation between genuine regression defects and intentional design iterations is labor-intensive and costly. To address this, we propose the first automated framework for semantic-level snapshot difference analysis. Our method leverages vision-language models (VLMs)—notably Gemma-3—to perform hierarchical classification of visual changes, enabling fine-grained root-cause identification (e.g., layout adjustments, styling updates, functional bugs). A configurable feature-flag mechanism ensures precise ground-truth labeling for constructing a high-quality, semantically annotated difference dataset. Experimental results show that a 12B-parameter VLM achieves 84.3% recall in root-cause classification; a 4B variant meets stringent CI/CD latency requirements. This work substantially reduces manual review effort and advances intelligent UI testing toward semantic awareness.

Automate analysis of UI snapshot test failuresImprove accuracy in identifying test failure causesReduce manual effort in distinguishing UI changes

Automated Test Case Repair Using Language Models

Jan 12, 2024
AS
Ahmadreza Saboor Yaraghi
🏛️ University of Ottawa | Carleton University | University of Limerick

Software evolution frequently causes test failures, leading to high maintenance costs and low efficiency. This paper proposes TaRGet, the first framework to formalize test repair as a context-aware code translation task. Leveraging pre-trained models such as CodeT5 and CodeLlama, TaRGet automates repair via failure-context extraction, error-pattern-aware input construction, and two-stage fine-tuning. Its key contributions are: (1) a novel formalization of test repair as translation; (2) TaRBench—the first large-scale, empirically grounded benchmark comprising over 45K real-world test repairs; and (3) a reliability-prediction guidance mechanism, empirically validated for cross-project cold-start generalization. On TaRBench, TaRGet achieves a 66.1% exact-match repair rate—significantly outperforming state-of-the-art baselines—without requiring any project-specific training data.

Maintenance CostQuality AssuranceSoftware Testing

Latest Papers

What's happening recently
View more

This work addresses the challenge that bug reports in open-source projects often lack reproducible tests, hindering effective repair. The authors propose a multi-stage agent framework that decomposes test reproduction into four phases: defect localization, root cause analysis, test planning, and test generation. For the first time, this approach integrates tool-augmented mechanisms and task decomposition strategies, leveraging code-text graph retrieval, runtime environment interaction, and large language models to achieve repository-level code understanding and flexible test construction. Evaluated on SWT-bench-lite and SWT-bench-verified, the method achieves reproduction success rates of 58.43% and 70.30%, respectively—substantially outperforming existing approaches—while maintaining a low average cost of only $0.14 per instance and demonstrably enhancing downstream repair performance.

bug reproductionissue reportslarge language models

This work addresses the limited ability of existing automated program repair methods to effectively leverage visual information in multimodal bug reports, particularly when dealing with heterogeneous images and extensive irrelevant regions, which hinders accurate fault localization and patch generation. To overcome this challenge, the paper proposes a novel visual software bug repair framework grounded in multimodal large language models (MLLMs), introducing two key innovations: an image-type-aware dynamic tool-calling chain and a test-time adaptive region-focusing mechanism with scalable attention. These components significantly enhance the model’s comprehension of visual inputs—such as UI screenshots—and improve fault localization precision. Evaluated on the SWE-bench Multimodal benchmark, the approach repairs 196 instances on the test set and 25 on the development set, outperforming the current best baseline by 10 and 11 instances, respectively, thereby demonstrating superior robustness and repair diversity.

Automated Program RepairGUI-based Software IssuesLarge Multimodal Language Models

This work addresses the reliability of automatic program repair (APR) evaluation by proposing the first rigorous reproducibility criteria for APR-oriented defect datasets. Applying these standards—encompassing automated test execution, static analysis, patch behavior comparison, and test suite adequacy checks—to the widely used Defects4J benchmark reveals significant shortcomings: 21.6% of its defects are unsuitable for APR evaluation, and an additional 7.1% suffer from substantially inadequate test suites, collectively rendering 28.7% (239 defects) prone to unreliable assessment. To promote more rigorous future research, the authors release the first open-source Java APR evaluation framework and call on the community to prioritize benchmark quality in empirical studies.

automated program repairbenchmark datasetDefects4J

Existing program repair benchmarks inadequately reflect real-world repository-level continuous integration (CI) scenarios, as they overlook critical challenges such as non-code artifacts, environmental dependencies, and workflow constraints. This work introduces the first repository-level repair benchmark grounded in actual GitHub Actions executions, validating patches through faithful replay of original CI workflows. The benchmark includes 567 CI failures meticulously annotated into 12 fine-grained error categories. Innovatively adopting end-to-end CI workflow re-execution as the patch validation criterion, it enables error-type-aware evaluation. By integrating log analysis, fault localization, and large language model–generated candidate patches, the approach achieves strong performance on tool-enforced errors like formatting and static checks, attaining an overall best repair success rate of 18.9%, while environment- and configuration-related issues remain notably challenging.

Automated Patch ValidationCI FailuresContinuous Integration

This study addresses the lack of systematic understanding regarding the impact of repair loop iteration counts in large language model (LLM)-based software engineering tasks, where prior work often relies on arbitrarily defined repair budgets. Through a cross-task (code generation, test generation, code translation) and cross-model empirical analysis, this work reveals—for the first time—a pronounced diminishing marginal returns phenomenon in iterative repair: performance gains are concentrated within the first 3–4 iterations, with negligible improvements thereafter. The findings underscore that the design of the repair workflow and feedback mechanisms exerts a far greater influence on repair efficacy than the choice of LLM itself. The authors advocate for treating repair budget as a critical experimental variable to ensure reliable, computationally efficient, and reproducible evaluation outcomes.

diminishing returnsiteration limitsLLM-based software engineering

Hot Scholars

SS

Sue Sentance

University of Cambridge
Computer Science EducationComputing EducationProgramming pedagogyTeaching CS
GF

Gordon Fraser

Professor of Computer Science, University of Passau
Software EngineeringSearch-based Software EngineeringSoftware TestingSpecification Mining
CM

Christopher M. Poskitt

Singapore Management University (SMU)
software engineeringsoftware testingformal methodscybersecurity
YN

Yannic Noller

Professor of Computer Science, Ruhr University Bochum (RUB)
Program RepairSoftware TestingSoftware QualitySoftware Engineering