🤖 AI Summary
This work addresses the limited effectiveness of current large language models (LLMs) in generating unit tests that reliably trigger real-world software defects. To overcome this limitation, the authors propose a knowledge-guided test generation framework that, for the first time, structures historical bug mechanisms into retrievable and executable feedback objectives. By integrating semantic representation, mechanism retrieval, and synthetic instantiation, the framework iteratively refines LLM-generated test cases to specifically target the triggering conditions and behavioral oracles of genuine defects. Experimental evaluation on Defects4J demonstrates that this approach significantly outperforms state-of-the-art baselines based on execution feedback, coverage criteria, mutation analysis, knowledge infusion, and search-based strategies, achieving a substantial improvement in real bug detection rates.
📝 Abstract
Large language models (LLMs) have opened new opportunities for unit test generation, but executable tests do not necessarily reveal real defects. This paper studies how historical real-bug mechanisms can be transformed into executable feedback targets for LLM-based unit test generation. The proposed framework constructs structural and semantic representations of real-bug records, retrieves mechanisms applicable to a focal method, and instantiates them as synthetic bugs that guide iterative test enhancement. We evaluate the approach on method-level real-bug detection tasks from Defects4J and show that mechanism-guided synthetic-bug feedback improves real-bug detection over execution-, coverage-, mutation-, knowledge-, and search-based baselines. The results suggest that organizing real-bug mechanisms as retrievable and executable feedback targets is an effective way to guide generated tests toward bug-triggering inputs and behavioral oracles.