coverage-guided fuzzing

Computer science students frequently exhibit persistent misconceptions about core testing concepts—particularly decision coverage and exception handling—leading to superficial test modifications that fail to improve code coverage. This study employs an automated testing feedback tool to systematically analyze multi-version student submissions and corresponding coverage changes, providing the first empirical identification of these two conceptual misunderstandings and their strong association with ineffective modification behaviors. Results show that over 70% of student test modifications yield no statistically significant coverage improvement, with errors predominantly occurring in branch decision logic and omitted exception-handling paths. The work reveals critical cognitive bottlenecks in testing education and delivers empirically grounded, actionable insights for designing targeted pedagogical interventions and intelligent, context-aware feedback tools.

When target code is missing or erroneous, generating reproducible test cases becomes challenging due to the absence of a correct oracle. To address this, this paper proposes an execution-feedback-driven test generation method that, without relying on a correct implementation, dynamically captures runtime behavioral deviations and guides test inputs toward conditions triggering SWE (Software Engineering) issues via repair-oriented constraint solving. Implemented in the custom tool e-Otter++, the approach overcomes the traditional limitation of requiring correct-code execution feedback. Evaluated on the TDD-Bench Verified benchmark, it achieves an average failure-to-pass (F2P) rate of 63%, significantly outperforming state-of-the-art techniques. Its core contribution is the first construction of a closed-loop execution feedback mechanism specifically designed for scenarios involving erroneous or missing code—enabling high-precision, robust reproduction of SWE issues through automatically generated test cases.

Defects in deep learning frameworks pose severe security risks in safety-critical domains; however, existing fuzzing techniques underutilize multi-source feedback and suffer from coarse granularity and low automation. This paper proposes FUEL—the first feedback-driven fuzzing framework leveraging dual large language model (LLM) agents: an *analysis LLM* performs fine-grained interpretation of coverage, crashes, and anomalies, while a *generation LLM* evolves high-diversity test cases based on this feedback, enabling closed-loop, synergistic feedback utilization. FUEL overcomes the static and unidirectional nature of conventional fuzzing feedback mechanisms. Evaluated on PyTorch and TensorFlow, FUEL identified 104 vulnerabilities, including 93 previously unknown ones; 47 have been patched, and 5 have received CVE identifiers.

This work addresses the inefficiency of traditional fuzzing in black-box or obfuscated binary programs where static instrumentation is infeasible and control-flow feedback is unavailable. The authors propose a dynamic feedback mechanism based on Execution Divergence Graphs (EDGs), which constructs control-flow-like structures at runtime by analyzing execution traces to precisely identify path divergences and avoid redundant exploration of loops. Requiring no static program information, the approach integrates divergence detection with an EDG-guided input mutation strategy. Evaluated on multiple obfuscated targets, it substantially outperforms blind fuzzers, demonstrating its effectiveness in non-instrumented settings. Furthermore, the framework is extensible to multidimensional feedback channels, such as power consumption, broadening its applicability in side-channel-aware fuzzing scenarios.

Existing fuzzing mutation strategies lack semantic awareness, leading to redundant test cases and insufficient exploration of deep program states. This paper proposes an LLM-guided, semantics-aware fuzzing framework: it integrates static control- and data-flow analysis with dynamic instrumentation to construct structured prompts that guide large language models in generating syntactically valid yet semantically diverse inputs; introduces a program-state-change–based semantic feedback mechanism—replacing conventional coverage-driven guidance; and incorporates embedded semantic similarity metrics and anomaly-aware semantic prioritization for seed selection. Evaluated on real-world projects—including libpng, tcpdump, and SQLite—the approach significantly accelerates first-bug discovery, enhances input semantic diversity, and increases the number of unique vulnerabilities identified, thereby overcoming the dual limitations of syntax-driven and coverage-oriented fuzzing paradigms.

This work addresses the high cost of manually crafting domain-specific fuzzing generators and the limited state-space coverage of generic approaches. The authors propose an intelligent agent framework powered by large language models that automatically synthesizes target-oriented input generators by interacting with a terminal and analyzing source code. The framework optionally incorporates predicate-level coverage feedback for iterative refinement. Experimental evaluation on seven real-world Java libraries demonstrates that the approach significantly outperforms human-written baselines on four of them. Notably, the synthesized generators embed sufficient structural and semantic logic to replace conventional coverage-guided mutation strategies, marking the first demonstration that AI agents can autonomously produce high-quality fuzzing generators without relying on external feedback signals.

This work addresses the limitations of traditional test case generation methods, which are labor-intensive and poorly scalable, as well as existing large language model (LLM)-based single-prompt approaches that often suffer from hallucination and redundancy, leading to insufficient branch coverage. The authors propose a two-stage automated test generation framework: in the first stage, static analysis extracts input constraints to generate initial test cases; in the second stage, dual agents leveraging line and branch feedback iteratively refine prompts based on uncovered metadata, while a redundancy-prevention caching mechanism avoids duplicate test generation. The approach is language-agnostic, supporting multiple programming languages such as C and Python, and achieves significant improvements in both line and branch coverage on standard benchmarks with linearly scalable execution time.

This work addresses the limited effectiveness of current large language models (LLMs) in generating unit tests that reliably trigger real-world software defects. To overcome this limitation, the authors propose a knowledge-guided test generation framework that, for the first time, structures historical bug mechanisms into retrievable and executable feedback objectives. By integrating semantic representation, mechanism retrieval, and synthetic instantiation, the framework iteratively refines LLM-generated test cases to specifically target the triggering conditions and behavioral oracles of genuine defects. Experimental evaluation on Defects4J demonstrates that this approach significantly outperforms state-of-the-art baselines based on execution feedback, coverage criteria, mutation analysis, knowledge infusion, and search-based strategies, achieving a substantial improvement in real bug detection rates.

This work addresses the challenge of reliably verifying large language model (LLM)-generated adaptive managers for collective adaptive systems (CAS) and ensuring their runtime correctness without manual code inspection. The authors propose an approach that integrates adaptive cycles with a feedback-driven Vibe-Coding mechanism, introducing Functional Constraints Logic (FCL)—a novel first-order temporal logic—to formally specify functional constraints. By synergistically combining architectural constraints, FCL-based verification, and runtime monitoring, the method generates fine-grained error feedback to iteratively guide the LLM toward correct implementations. Evaluated on the Dragon Hunt CAS case study, the approach produces effective adaptive managers within only a few iterations, substantially outperforming baseline methods that rely on coarse-grained metrics, thereby demonstrating the feasibility of fully automated verification and repair.