fuzz testing

In software testing, misalignment between business requirements and test cases impedes effective quality shift-left. To address this core challenge, we propose a high-level test case generation method prioritizing *requirement alignment*: (1) We construct the first industrial-scale dataset—BAlign—comprising requirement-aligned, executable test cases grounded in real-world business semantics; (2) We fine-tune open-source LLMs (LLaMA 3.1-8B, Mistral-7B) via supervised fine-tuning to automatically generate human-readable, executable test cases that comprehensively cover functional points and expected outcomes. Experimental results demonstrate that our fine-tuned models significantly outperform proprietary large language models (e.g., GPT-4o, Gemini) in both automated metrics and functional correctness. Human evaluation further confirms that the generated test cases exhibit high business interpretability and engineering practicality, effectively bridging the semantic gap between requirements and testing artifacts.

Scaling high-quality synthetic data generation for code large language models remains challenging—existing approaches rely on expensive, powerful teacher models and suffer from limited diversity and correctness. Method: This paper introduces the Case2Code task, the first scalable synthetic paradigm that frames program behavior induction (case-to-code) as an end-to-end pipeline: large language models generate diverse inputs; automated execution yields ground-truth outputs; and dynamic testing rigorously validates functional correctness—eliminating dependence on teacher models while enabling low-cost, high-diversity, high-fidelity code data synthesis. Contribution/Results: Empirical evaluation demonstrates that models trained on Case2Code data achieve significant improvements in both case-to-code generalization and standard benchmarks (HumanEval, MBPP), confirming the effectiveness and transferability of inductive synthetic data for code modeling.

This work addresses black-box program behavior modeling by proposing a reversible, differentiable, and constraint-aware, grammar-driven neural modeling framework. Methodologically, it generates input-output (I/O) pairs from formal grammars of input and output languages, and employs a lightweight (<6.3M-parameter) sequence-to-sequence model to cast program I/O mapping as a bidirectional neural machine translation task—enabling both forward prediction and backward inference—while supporting fine-grained behavioral constraints and fault- or coverage-guided input synthesis. Its key contribution is the first end-to-end joint modeling of reversibility, differentiability, and syntactic consistency in program behavior models. Evaluated on structured tasks such as Markdown and HTML generation, the framework achieves 95.4% accuracy and a BLEU score of 0.98±0.04, significantly outperforming existing irreversible or syntax-agnostic approaches.

This work addresses the insufficient coverage of boundary and subtle error cases in existing code generation benchmarks, which often misclassify incorrect solutions as correct. To mitigate this, the authors propose an agent-based automated adversarial testing framework that emulates the “Hack” mechanism from competitive programming. The framework employs multiple strategies—including stress testing, anti-hash attacks, and logic-directed attacks—to generate targeted adversarial test cases. It further incorporates a self-calibration phase, where self-generated probes iteratively refine the validator and checker components. Experimental results demonstrate that the approach significantly improves the true negative rate (TNR), effectively identifying previously misjudged erroneous programs. Moreover, the generated test cases, when used as training data, enhance the performance of reinforcement learning models on benchmarks such as LiveCodeBench.

Neural test generation suffers from weak semantic validity due to scarce and insufficiently diverse training data—particularly for emerging programming languages. To address this, we propose FuzzAug, the first data augmentation framework for neural test generation that integrates fuzzing-inspired heuristic mutation. FuzzAug jointly optimizes input diversity and program semantic fidelity by (1) generating candidate tests via fuzzing-based mutation, (2) fine-tuning large language models (LLMs) on augmented data, (3) validating outputs via compilation, and (4) iteratively refining candidates using branch coverage feedback. This enables dynamic co-optimization between fuzzing-driven test generation and LLM-based synthesis. Empirical evaluation demonstrates that FuzzAug improves assertion accuracy by 5%, compilation success rate by over 10%, and branch coverage of generated test functions by 5 percentage points—substantially enhancing LLMs’ capability to produce high-quality, high-coverage unit tests.

Existing code clone detection models are trained on clean data but exhibit poor robustness against semantic-preserving transformations (e.g., refactoring, minification, formatting, compiler optimizations) prevalent in real-world code, leading to train-deploy distribution shift. To address this, we propose Auto-SPT—a novel framework that leverages large language models (LLMs) to automatically discover, compose, and formally verify diverse semantic-preserving transformations for synthesizing highly robust training data. Its core contributions are: (1) LLM-driven automatic construction and strength-aware modeling of transformations; (2) a scalable composition mechanism enabling combinatorial transformation generation; and (3) a task-specific adversarial data augmentation paradigm tailored for clone detection. Experiments show that Auto-SPT–generated transformations substantially degrade state-of-the-art models’ accuracy (average drop of 32.7%), while using them for augmentation improves model F1 scores by up to 18.4% across diverse perturbations, effectively bridging the robustness gap in practical deployment scenarios.

This work addresses the limitations of existing vulnerability detection benchmarks, which are predominantly confined to the function level and thus fail to capture cross-procedural vulnerabilities prevalent in real-world scenarios, while manually curated repository-scale datasets suffer from limited scalability. To overcome these challenges, the paper proposes an automated benchmark generation framework that injects realistic vulnerabilities into genuine code repositories and synthesizes reproducible proofs of vulnerability (PoVs), thereby constructing a large-scale, precisely labeled repository-level vulnerability dataset. This framework represents the first scalable approach to automatically generating repository-level vulnerability benchmarks. Furthermore, it introduces an adversarial co-evolution mechanism, enabling dynamic interaction between vulnerability injection and detection agents under realistic constraints, which significantly enhances the robustness and practical utility of vulnerability detection models.

Gray-box fuzzing suffers from low-quality initial seed sets (ICS) and a disjointed workflow between seed generation and corpus minimization. Method: This paper proposes an LLM-driven multi-agent collaborative framework that unifies high-coverage, high-bug-triggering ICS generation. It introduces a novel execution-free predictive coverage assessment mechanism, where a dedicated LLM agent dynamically evaluates seed utility in real time; seed generation and corpus minimization are integrated into a single end-to-end process. Contribution/Results: Experiments demonstrate that our approach achieves higher code coverage and detects more runtime errors using fewer test cases than state-of-the-art baselines. Both time efficiency and coverage efficiency are significantly improved, validating the effectiveness of LLM-guided, agent-coordinated seed optimization.

Existing competitive programming benchmarks rely on manual problem authoring, suffering from data contamination and poor scalability. To address these limitations, this paper proposes UniCode—a novel framework that integrates large language models (LLMs) with biologically inspired evolutionary mechanisms to enable fully automated, high-quality algorithmic problem generation. UniCode enhances problem diversity via single-problem expansion and intra- and cross-category fusion. It introduces a stimulus-driven test-case generation pipeline—requiring no reference solutions—that combines small-scale brute-force grounding with large-scale consensus-based validation to ensure robustness and comprehensive coverage. We construct a contamination-resistant evaluation benchmark comprising 492 problems. Evaluating 19 mainstream LLMs reveals that even the strongest model, o4-mini, achieves only a 70.3% pass rate, demonstrating UniCode’s high difficulty and strong discriminative power.

Existing code corpora commonly suffer from uncontrolled semantics, structural disorder, ambiguous difficulty levels, lack of standardized references, and training data contamination. This work proposes a language-agnostic, on-demand generation framework based on a JSON intermediate representation, employing a deoptimizing transpiler to produce deliberately redundant and fully flattened programs in five languages (Python, JavaScript, Go, Java, and C++), with correctness guaranteed via oracle validation. The resulting dataset is intrinsically correct, contamination-resistant, and annotated with self-contained difficulty labels capturing both intrinsic and incidental complexity, enabling fine-grained control over code readability and complexity. Experiments demonstrate that the generated data effectively reflects a difficulty gradient—model accuracy drops to zero as intrinsic difficulty increases—and exhibits minimal performance discrepancy (<0.012) between development and novel test sets, with self-annotations providing significantly greater gains for weaker models than for stronger ones.