program synthesis

This work investigates the feasibility and underlying mechanisms of automatic program repair (APR) using large language models (LLMs) without fine-tuning, via a code→natural language→code round-trip translation (RTT) paradigm. We propose RTT-APR—the first fine-tuning-free APR framework—whose core mechanism is leveraging the pre-trained model’s implicit prior over correct code to suppress defect-induced noise through “regression toward the mean.” We evaluate RTT-APR across eight LLMs—including GPT-3.5 and GPT-4—and four Java benchmarks, notably HumanEval-Java. GPT-4 repairs 101 out of 164 buggy programs using RTT, including 46 defects entirely missed by all existing fine-tuned LLM-based APR methods. Our results demonstrate that RTT-APR is a lightweight, effective, and mechanistically interpretable APR technique, offering a novel, parameter-efficient alternative to fine-tuning–dependent approaches.

This work addresses the industrial ineffectiveness of automated program repair (APR) and the misalignment between academic benchmarks and real-world defects. We propose a novel approach that leverages formal specifications—expressed in the Java Modeling Language (JML)—to guide large language models (LLMs) in generating high-coverage unit tests. By embedding specifications into the test-generation pipeline and integrating contract automata with human feedback, our method significantly improves detection of logic errors, string manipulations, and exception-handling edge cases. Evaluation shows that our technique outperforms state-of-the-art APR methods on standard benchmarks; JML specifications prove particularly valuable for exposing complex semantic errors; yet passing generated tests does not guarantee repair correctness. The study reveals a fundamental gap between academic evaluation criteria and industrial requirements, offering both a scalable technical pathway and empirical evidence toward practical, specification-guided program repair.

Ambiguity in natural-language requirements frequently leads large language models (LLMs) to generate incorrect code. To address this, we propose SpecFix—a fully automated ambiguity resolution method that requires no LLM metacognitive capabilities. SpecFix first models the distribution of program interpretations induced by an LLM’s responses to the original requirement, using program testing and automated repair. It then infers a revised, minimal, and verifiable specification by reversely analyzing distributional shifts and iteratively contracting the requirement space under logical constraints. Crucially, SpecFix decouples ambiguity resolution into two novel phases: (1) modeling the program-interpretation distribution and (2) constraint-driven, contraction-based specification inference—thereby eliminating reliance on self-reflection. Evaluated on HumanEval+ and MBPP+, SpecFix improves Pass@1 by 4.3% on average across GPT-4o, DeepSeek-V3, and Qwen2.5-Coder, and boosts majority-vote solution rates by 3.4%.

This work proposes a human-AI collaborative paradigm for formal software specification that mitigates the traditional barriers to industrial adoption—namely, the notational complexity and high expertise threshold—while preserving the benefits of early error detection and explicit invariants. The approach employs an intermediate language blending natural language with lightweight LaTeX mathematical notation, enabling AI-assisted review, refinement, and code generation. Crucially, it distinguishes between components requiring rigorous formalization and those amenable to flexible treatment. By deeply integrating AI into the specification authoring and verification workflow, this method achieves “correct-by-construction” development in a case study on organizational knowledge growth simulation, significantly reducing costs while ensuring early validation and design correctness.

Software evolution frequently causes test failures, leading to high maintenance costs and low efficiency. This paper proposes TaRGet, the first framework to formalize test repair as a context-aware code translation task. Leveraging pre-trained models such as CodeT5 and CodeLlama, TaRGet automates repair via failure-context extraction, error-pattern-aware input construction, and two-stage fine-tuning. Its key contributions are: (1) a novel formalization of test repair as translation; (2) TaRBench—the first large-scale, empirically grounded benchmark comprising over 45K real-world test repairs; and (3) a reliability-prediction guidance mechanism, empirically validated for cross-project cold-start generalization. On TaRBench, TaRGet achieves a 66.1% exact-match repair rate—significantly outperforming state-of-the-art baselines—without requiring any project-specific training data.

This work addresses the challenges of ambiguity and validity verification in automatically translating natural language assertions into formal, executable specifications—a task traditionally reliant on error-prone manual effort. The paper proposes Monty, a novel framework that leverages large language models to generate candidate formal assertions and introduces an innovative combination of code-based testing and consistency scoring to automatically select high-quality translations. Evaluated on 541 tasks, Monty achieves up to a 20-percentage-point improvement in average precision over baseline approaches that directly use large language models for translation, substantially enhancing the accuracy and reliability of automated formalization.

Existing large code models struggle to generate executable intermediate formal specifications, limiting precise verification and repair of program behavioral errors. This work proposes SpecCoder, a novel framework that focuses on generating executable inline assertions at critical program locations, thereby transforming static annotations into verifiable evidence. SpecCoder employs verification-guided training, fine-tuning the Qwen2.5-Coder series models using correct programs, behavioral mutants, and multi-round specification refinement trajectories. Evaluated on the HumanExec benchmark, SpecCoder substantially improves the correctness (+55.8%), completeness (+358.1%), and assertion validity (+26.6%) of inline specifications, significantly enhancing program verification and repair capabilities.

Traditional mutation testing struggles to generate mutants that exhibit subtle semantic differences and closely resemble real-world programming faults, thereby limiting test effectiveness. This work proposes a novel approach that, for the first time, integrates large language model–driven round-trip translation between code and natural language intent into mutation testing. By leveraging translation discrepancies and controlled perturbations of intended behavior, the method generates high-quality mutants with nuanced semantic variations. Empirical evaluation on 40 real faulty methods demonstrates that the proposed technique—referred to as RTM—significantly improves fault detection rates using substantially smaller test suites: with only 4 and 30 test cases, RTM detects on average 4× and 1.7× more faults, respectively, than conventional approaches, confirming its efficiency and practicality.

This work addresses the challenge of statically verifying semantic consistency between natural language business requirements and their code implementations. It proposes a two-stage, runtime-free approach: first leveraging large language models to extract structured rules from requirements while identifying ambiguous or contradictory statements, and then performing static code auditing based on this intermediate representation. By integrating natural language processing with static analysis, the method mitigates hallucination and context loss in large models through rule structuring, enabling requirement-aware early validation. Evaluated on an automotive cybersecurity case study, the approach successfully detects semantic deviations, offers a novel solution to the test oracle problem, and significantly enhances left-shifted verification capabilities.

Existing direct code-to-code transformation approaches often suffer from semantic drift, implicit behavioral changes, and loss of traceability. To address these issues, this work proposes a specification-based Code2Text2Code refactoring framework that first translates source code into a neutral textual specification before generating target code. The approach integrates abstract syntax tree (AST) and dependency graph analysis, semantic-aware code chunking, retrieval-augmented generation, and DSPy-based prompt tuning, further enhanced by iterative validation and graph-based formal verification. This pipeline ensures high-fidelity semantic preservation and controllable evolution during code transformation. Experimental results demonstrate that the proposed method significantly reduces transformation loss and substantially improves semantic consistency, interface stability, and cross-language traceability of the refactored code.