Generating REST API Tests With Descriptive Names

2025-12-01

AI Summary

Automated REST API test case generation often yields meaningless identifiers (e.g., `test0`, `test1`), severely impairing readability and maintainability. To address this, we propose three lightweight, deterministic, and external-API-free naming strategies that integrate rule-based heuristics with large language model (LLM) techniques to produce semantically rich, syntactically consistent, and descriptive test names. Unlike black-box LLMs such as GPT-3.5, our approach ensures enhanced security, full controllability, and significantly lower computational overhead—while achieving clarity scores comparable to GPT-4o. Empirical evaluation demonstrates substantial improvements in test understandability; an industrial deployment at Volkswagen AG further validates its practical efficacy and operational value. This work establishes a robust, efficient, and trustworthy naming paradigm for automated testing engineering.

Technology Category

• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Planning, Routing, and Scheduling: Planning with Language Models
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Automated test generation has become a key technique for ensuring software quality, particularly in modern API-based architectures. However, automatically generated test cases are typically assigned non-descriptive names (e.g., test0, test1), which reduces their readability and hinders their usefulness during comprehension and maintenance. In this work, we present three novel deterministic techniques to generate REST API test names. We then compare eight techniques in total for generating descriptive names for REST API tests automatically produced by the fuzzer EvoMaster, using 10 test cases generated for 9 different open-source APIs. The eight techniques include rule-based heuristics and large language model (LLM)-based approaches. Their effectiveness was empirically evaluated through two surveys (involving up to 39 people recruited via LinkedIn). Our results show that a rule-based approach achieves the highest clarity ratings among deterministic methods, performs on par with state-of-the-art LLM-based models such as Gemini and GPT-4o, and significantly outperforms GPT-3.5. To further evaluate the practical impact of our results, an industrial case study was carried out with practitioners who actively use EvoMaster at Volkswagen AG. A developer questionnaire was then carried out based on the use of EvoMaster on four different APIs by four different users, for a total of 74 evaluated test cases. Feedback from practitioners further confirms that descriptive names produced by this approach improve test suite readability. These findings highlight that lightweight, deterministic techniques can serve as effective alternatives to computationally expensive and security-sensitive LLM-based approaches for automated system-level test naming, providing a practical step toward more developer-friendly API test generation.

Problem

• Generates descriptive names for REST API tests
• Compares rule-based and LLM-based naming techniques
• Improves test readability for comprehension and maintenance

Innovation

• Rule-based heuristics generate descriptive REST API test names
• Deterministic techniques outperform GPT-3.5 and match advanced LLMs
• Lightweight methods improve test readability without security concerns