Score
Systematic methods for reproducing, isolating, and fixing faults in prompts, code, or visualizations, including iterative refinement, failure analysis, triage, and deciding corrective actions or rollbacks based on test outcomes.
In industrial quality inspection, anomaly detection suffers from poor robustness due to high noise levels and sparse defective samples. To address this, we propose Iterative Refinement of Pseudo-labels (IRP), a self-supervised method that alternately evaluates sample credibility and removes misleading instances under feature-space consistency constraints—effectively purifying the training set dynamically without human annotations and generating high-fidelity self-supervised signals. IRP introduces the novel paradigm of “iterative data refinement,” significantly enhancing model robustness against label noise and cross-domain generalization capability. Evaluated on KSDD2 and MVTec AD benchmarks, IRP consistently outperforms existing unsupervised and self-supervised methods. Notably, under high-noise conditions, it achieves substantial improvements in detection accuracy and reduces false positive rates by over 25%.
Existing LLM-based automated program repair (APR) methods rely on coarse-grained test feedback and lack fine-grained error attribution, resulting in low repair efficiency and severe error propagation. To address this, we propose TokenRepair—the first dual-layer APR framework integrating internal reflection and external feedback. It first identifies potentially erroneous tokens via context-aware, token-level uncertainty analysis; then performs precise patch generation through chain-of-thought-guided local rewriting and quality-aware iterative optimization—avoiding global regeneration to significantly improve repair stability and accuracy. Evaluated on Defects4J 1.2 and HumanEval-Java, TokenRepair successfully repairs 88 and 139 bugs, respectively, outperforming state-of-the-art methods by 8.2%–34.9%. These results empirically validate the effectiveness and generalizability of our token-level fine-grained reflection mechanism.
This paper addresses the challenge of automated repair of C programs by proposing the first closed-loop framework that integrates spectrum-based fault localization (SBFL) with large language model (LLM)-driven chain-of-thought (CoT) reasoning. Methodologically, the approach employs runtime test feedback to drive iterative refinement: SBFL initially identifies suspicious statements; an LLM generates patches using structured CoT prompting; and execution feedback guides successive improvements until all tests pass. The key contribution lies in the explicit coupling of statistical program analysis with symbolic LLM reasoning—enhancing fault attribution accuracy and mitigating recurrent patch failures. Evaluated on the Codeflaws benchmark (3,902 defects), the method achieves a repair accuracy of 44.93%, outperforming the GPT-4+CoT baseline by 3.61%, thereby validating the efficacy of analysis-guided iterative reasoning for program repair.
UI snapshot testing suffers from high false-positive rates due to frequent interface changes, and manual differentiation between genuine regression defects and intentional design iterations is labor-intensive and costly. To address this, we propose the first automated framework for semantic-level snapshot difference analysis. Our method leverages vision-language models (VLMs)—notably Gemma-3—to perform hierarchical classification of visual changes, enabling fine-grained root-cause identification (e.g., layout adjustments, styling updates, functional bugs). A configurable feature-flag mechanism ensures precise ground-truth labeling for constructing a high-quality, semantically annotated difference dataset. Experimental results show that a 12B-parameter VLM achieves 84.3% recall in root-cause classification; a 4B variant meets stringent CI/CD latency requirements. This work substantially reduces manual review effort and advances intelligent UI testing toward semantic awareness.
Software evolution frequently causes test failures, leading to high maintenance costs and low efficiency. This paper proposes TaRGet, the first framework to formalize test repair as a context-aware code translation task. Leveraging pre-trained models such as CodeT5 and CodeLlama, TaRGet automates repair via failure-context extraction, error-pattern-aware input construction, and two-stage fine-tuning. Its key contributions are: (1) a novel formalization of test repair as translation; (2) TaRBench—the first large-scale, empirically grounded benchmark comprising over 45K real-world test repairs; and (3) a reliability-prediction guidance mechanism, empirically validated for cross-project cold-start generalization. On TaRBench, TaRGet achieves a 66.1% exact-match repair rate—significantly outperforming state-of-the-art baselines—without requiring any project-specific training data.
This work addresses the challenge that bug reports in open-source projects often lack reproducible tests, hindering effective repair. The authors propose a multi-stage agent framework that decomposes test reproduction into four phases: defect localization, root cause analysis, test planning, and test generation. For the first time, this approach integrates tool-augmented mechanisms and task decomposition strategies, leveraging code-text graph retrieval, runtime environment interaction, and large language models to achieve repository-level code understanding and flexible test construction. Evaluated on SWT-bench-lite and SWT-bench-verified, the method achieves reproduction success rates of 58.43% and 70.30%, respectively—substantially outperforming existing approaches—while maintaining a low average cost of only $0.14 per instance and demonstrably enhancing downstream repair performance.
This work addresses the limited ability of existing automated program repair methods to effectively leverage visual information in multimodal bug reports, particularly when dealing with heterogeneous images and extensive irrelevant regions, which hinders accurate fault localization and patch generation. To overcome this challenge, the paper proposes a novel visual software bug repair framework grounded in multimodal large language models (MLLMs), introducing two key innovations: an image-type-aware dynamic tool-calling chain and a test-time adaptive region-focusing mechanism with scalable attention. These components significantly enhance the model’s comprehension of visual inputs—such as UI screenshots—and improve fault localization precision. Evaluated on the SWE-bench Multimodal benchmark, the approach repairs 196 instances on the test set and 25 on the development set, outperforming the current best baseline by 10 and 11 instances, respectively, thereby demonstrating superior robustness and repair diversity.
This work addresses the reliability of automatic program repair (APR) evaluation by proposing the first rigorous reproducibility criteria for APR-oriented defect datasets. Applying these standards—encompassing automated test execution, static analysis, patch behavior comparison, and test suite adequacy checks—to the widely used Defects4J benchmark reveals significant shortcomings: 21.6% of its defects are unsuitable for APR evaluation, and an additional 7.1% suffer from substantially inadequate test suites, collectively rendering 28.7% (239 defects) prone to unreliable assessment. To promote more rigorous future research, the authors release the first open-source Java APR evaluation framework and call on the community to prioritize benchmark quality in empirical studies.
Existing program repair benchmarks inadequately reflect real-world repository-level continuous integration (CI) scenarios, as they overlook critical challenges such as non-code artifacts, environmental dependencies, and workflow constraints. This work introduces the first repository-level repair benchmark grounded in actual GitHub Actions executions, validating patches through faithful replay of original CI workflows. The benchmark includes 567 CI failures meticulously annotated into 12 fine-grained error categories. Innovatively adopting end-to-end CI workflow re-execution as the patch validation criterion, it enables error-type-aware evaluation. By integrating log analysis, fault localization, and large language model–generated candidate patches, the approach achieves strong performance on tool-enforced errors like formatting and static checks, attaining an overall best repair success rate of 18.9%, while environment- and configuration-related issues remain notably challenging.
This study addresses the lack of systematic understanding regarding the impact of repair loop iteration counts in large language model (LLM)-based software engineering tasks, where prior work often relies on arbitrarily defined repair budgets. Through a cross-task (code generation, test generation, code translation) and cross-model empirical analysis, this work reveals—for the first time—a pronounced diminishing marginal returns phenomenon in iterative repair: performance gains are concentrated within the first 3–4 iterations, with negligible improvements thereafter. The findings underscore that the design of the repair workflow and feedback mechanisms exerts a far greater influence on repair efficacy than the choice of LLM itself. The authors advocate for treating repair budget as a critical experimental variable to ensure reliable, computationally efficient, and reproducible evaluation outcomes.