ATTest: Agent-Driven Tensor Testing for Deep Learning Library Modules

2026-02-15

AI Summary

This work addresses the challenge of automatically generating unit tests for deep learning libraries, which is hindered by complex numerical semantics and implicit tensor constraints. Existing approaches either suffer from semantic blind spots or are limited by context length and generation instability. To overcome these issues, we propose ATTest, a novel framework that introduces an agent-driven workflow for module-level test generation through a seven-stage pipeline. ATTest integrates constraint extraction, an iterative “generate–verify–repair” mechanism, and collaboration with large language models to effectively bridge semantic gaps and alleviate context saturation. Evaluated on PyTorch and TensorFlow, ATTest achieves average branch coverage of 55.60% and 54.77%, respectively, significantly outperforming state-of-the-art baselines such as PynguinML.

Technology Category

• Machine Learning: Deep Generative Models & Autoencoders
• Natural Language Processing: Generation
• Constraint Satisfaction and Optimization: Satisfiability Modulo Theories

Application Category

• Search and Retrieval-Augmented AI: Agentic search
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

The unit testing of Deep Learning (DL) libraries is challenging due to complex numerical semantics and implicit tensor constraints. Traditional Search-Based Software Testing (SBST) often suffers from semantic blindness, failing to satisfy the constraints of high-dimensional tensors, whereas Large Language Models (LLMs) struggle with cross-file context and unstable code modifications. This paper proposes ATTest, an agent-driven tensor testing framework for module-level unit test generation. ATTest orchestrates a seven-stage pipeline, which encompasses constraint extraction and an iterative"generation-validation-repair"loop, to maintain testing stability and mitigate context-window saturation. An evaluation on PyTorch and TensorFlow demonstrates that ATTest significantly outperforms state-of-the-art baselines such as PynguinML, achieving an average branch coverage of 55.60% and 54.77%, respectively. The results illustrate how agent-driven workflows bridge the semantic gap in numerical libraries while ensuring auditable test synthesis. Source code: https://github.com/iSEngLab/ATTest.git

Problem

• unit testing
• deep learning libraries
• tensor constraints
• semantic blindness
• LLM limitations

Innovation

• agent-driven testing
• tensor constraint extraction
• iterative generation-validation-repair
• deep learning library testing
• unit test synthesis