unit testing

This work addresses the industrial ineffectiveness of automated program repair (APR) and the misalignment between academic benchmarks and real-world defects. We propose a novel approach that leverages formal specifications—expressed in the Java Modeling Language (JML)—to guide large language models (LLMs) in generating high-coverage unit tests. By embedding specifications into the test-generation pipeline and integrating contract automata with human feedback, our method significantly improves detection of logic errors, string manipulations, and exception-handling edge cases. Evaluation shows that our technique outperforms state-of-the-art APR methods on standard benchmarks; JML specifications prove particularly valuable for exposing complex semantic errors; yet passing generated tests does not guarantee repair correctness. The study reveals a fundamental gap between academic evaluation criteria and industrial requirements, offering both a scalable technical pathway and empirical evidence toward practical, specification-guided program repair.

This study addresses the imbalance in the test pyramid—characterized by an overreliance on coarse-grained integration and system tests, which leads to difficulties in fault localization and slow execution—by proposing, for the first time, a method to automatically generate unit tests from existing integration tests. The approach combines static and dynamic analysis to automatically isolate component dependencies and enhance coverage at the unit level. Implemented as a Node.js tool and evaluated on twelve open-source JavaScript projects, the technique produces high-quality unit tests that significantly improve test suite structure, thereby increasing both testing efficiency and maintainability.

To address the insufficient speed, reliability, and maintainability of testing in modern software systems, this paper designs and implements a modular automated testing framework that deeply integrates Cucumber-BDD with Java. The framework introduces a novel natural-language-driven test design and engineering implementation co-development mechanism, supporting dynamic environment adaptation, reusable component-based architecture, and end-to-end automated reporting with closed-loop feedback. It integrates Selenium, TestNG, Maven, and Jenkins to enable seamless embedding into CI/CD pipelines. Empirical evaluation demonstrates a reduction of manual testing effort by over 40%, a 35% improvement in defect detection rate, and a 50% decrease in script maintenance cost. These outcomes significantly enhance agility in iterative development and streamline multi-environment one-click deployment efficiency.

This work addresses the challenge that rapid software development often compromises code maintainability, thereby hindering safe AI-assisted refactoring. To mitigate this, the authors propose an iterative refactoring approach that integrates large language models with human oversight. The method first leverages a code-specialized large language model to automatically generate high-coverage unit tests that capture existing program behavior. Subsequently, developers guide test-driven refactoring, while branch coverage metrics are used to constrain and validate model-generated outputs. Empirical evaluation demonstrates that the approach produces nearly 16,000 lines of reliable test code within hours, achieving up to 78% branch coverage on critical modules. This significantly reduces regression risk during large-scale refactoring and enhances the reliability and practicality of AI-assisted code restructuring.

Existing code-level formal verification tools scale poorly to large-scale software, while mainstream unit-level verification relies heavily on manual effort, often missing critical defects. This paper proposes the “Unit Proof Framework” research agenda—the first systematic definition of a unit verification paradigm supporting automated decoupling and independent verification of code units. Methodologically, it integrates formal verification, program analysis, modular verification, and automated toolchain design, with deep alignment to industrial development practices (e.g., AWS workflows). Its core contributions include: (1) establishing a scalable, engineering-friendly unit verification methodology; (2) characterizing a taxonomy of key technical challenges; (3) overcoming bottlenecks inherent in manual verification; and (4) significantly improving early detection of code-level defects. Collectively, this work lays the theoretical foundation and provides a practical technical pathway for building high-assurance, deployable automated verification infrastructure.

This study investigates the gap between real-world developer testing practices and automated test generation (ATG) techniques. Method: We conduct a large-scale empirical analysis of 1.7 million manually written Java test cases from open-source projects, characterizing their key features across five dimensions—test scope, fixture design, assertion patterns, input types, and mocking usage—and benchmarking them against two state-of-the-art ATG tools, EvoSuite and Randoop. Contribution/Results: Our analysis reveals, for the first time at scale, that the majority of human-written tests incorporate complex contextual dependencies, domain-specific semantic assertions, and fine-grained mocking logic—capabilities largely absent in current ATG tools. The study establishes the first large-scale empirical benchmark for aligning ATG with industrial practice and provides concrete, actionable directions for improving ATG’s fidelity to real-world testing needs.

This work addresses the challenge of statically verifying semantic consistency between natural language business requirements and their code implementations. It proposes a two-stage, runtime-free approach: first leveraging large language models to extract structured rules from requirements while identifying ambiguous or contradictory statements, and then performing static code auditing based on this intermediate representation. By integrating natural language processing with static analysis, the method mitigates hallucination and context loss in large models through rule structuring, enabling requirement-aware early validation. Evaluated on an automotive cybersecurity case study, the approach successfully detects semantic deviations, offers a novel solution to the test oracle problem, and significantly enhances left-shifted verification capabilities.

This paper addresses the problem that large language models (LLMs) generate unrealistic test inputs and unreliable assertions for unit testing due to insufficient semantic understanding. To tackle this, the authors systematically survey 115 studies and propose the first unified, lifecycle-spanning taxonomy for LLM-based unit test generation. Methodologically, they model LLMs as stochastic generators requiring engineering constraints, categorizing techniques into five dimensions: context enhancement, prompt engineering, assertion synthesis, iterative validation and repair, and pre-/post-processing quality assurance. Results show that 89% of existing works rely on prompt engineering; iterative validation significantly improves compilation and execution pass rates; yet defect detection capability remains weak, and standardized evaluation benchmarks are lacking. The study further identifies key research trajectories toward autonomous testing agents and hybrid testing systems.

This work addresses the challenge of applying equivalence class partitioning—a testing requirement under ISO 26262—to legacy embedded firmware in the absence of complete specification documents. The authors propose a binary-level method that automatically infers output-oriented equivalence classes by reconstructing control flow and performing guided symbolic execution to analyze function behavior. Execution paths are clustered based on observable outputs, such as return values and output parameters, and the resulting equivalence classes are represented in a human-readable form to support test design. To the best of the authors’ knowledge, this is the first approach capable of inferring equivalence classes directly from binaries without source code or documentation for safety-critical embedded software. Industrial case studies demonstrate that the inferred classes align closely with expert expectations and offer both high readability and practical utility, effectively aiding functional comprehension and compliance testing of legacy firmware.

Detecting subtle, specification-omitted bugs in Boogie—a widely used intermediate verification language—is challenging due to the incompleteness of existing formal models. Method: We propose BCC, a lightweight model-based testing technique grounded in executable operational semantics. BCC integrates the PLT Redex framework with a small, deterministic subset of Boogie’s operational semantics to automatically generate random programs; it then identifies bugs by comparing semantic simulation results against actual Boogie verification outcomes. Contribution/Results: BCC breaks from conventional reliance on full formal models by leveraging executable semantics to drive randomized testing—thereby efficiently exercising complex, non-canonical implementation paths in the toolchain. In evaluation, BCC generated 3 million test programs and uncovered completeness violations in 2% of them. These findings demonstrate BCC’s effectiveness and practicality for ensuring the reliability of verification tools themselves.