Agent-Based Test Assertion Generation via Diverse Perspective Aggregation

2026-08-06

AI Summary

Current large language model (LLM)-driven approaches to test assertion generation suffer from low accuracy, reliance on oversampling, and high sensitivity to randomness. This work proposes AssertMate, a novel framework that introduces multi-agent collaboration and diverse perspective aggregation into assertion generation for the first time. AssertMate identifies assertion targets via static analysis and employs multiple agents—each leveraging type-aware heuristics, code generation, retrieval-augmented generation (RAG), and chain-of-thought (CoT) reasoning—to predict expected values from complementary viewpoints. An LLM-as-a-Judge mechanism then selects the optimal assertion. Experimental results demonstrate that AssertMate significantly outperforms existing methods on Defects4J, achieving substantial improvements in compilation success rate, test pass rate, and defect detection capability. When integrated with EvoSuite, it also attains higher mutation coverage and kill counts.

Abstract

Test assertions are critical elements of unit tests, serving as checkpoints to validate expected behavior and ensure software correctness. Numerous techniques have been proposed to automate assertion generation, with recent progress notably driven by large language models (LLMs). Despite the promise, existing approaches such as ChatAssert suffer from modest accuracy, heavy reliance on oversampling, and vulnerability to model randomness due to one-shot prompting. To address these limitations, we propose AssertMate, a novel agent-based assertion generation framework that enhances the quality and reliability of LLM-generated assertions through three key components: (1) actual value construction that identifies assertion targets via static analysis and type-aware heuristics; (2) multi-perspective expected value prediction using code generation, retrieval-augmented generation (RAG), and chain-of-thought (CoT) reasoning agents; and (3) an LLM-as-a-Judge collaboration mechanism to select the most appropriate assertion. Evaluation on the Defects4J benchmark demonstrates that AssertMate significantly outperforms state-of-the-art techniques in compilation success and pass rates, along with substantially higher bug detection capabilities. Integration with EvoSuite further validates AssertMate's practicality, yielding superior mutation coverage and kill counts. Ablation studies reveal that each of the three components makes a significant and complementary contribution to the overall performance. This work affirms the great potential of aggregating diverse perspectives to enhance the effectiveness of LLM-based assertion generation.

Problem

• test assertion generation
• large language models
• software testing
• LLM randomness
• automated testing

Innovation

• agent-based assertion generation
• retrieval-augmented generation
• chain-of-thought reasoning
• LLM-as-a-Judge
• static analysis