RESTifAI: LLM-Based Workflow for Reusable REST API Testing

2025-12-09

AI Summary

Existing REST API testing tools primarily focus on detecting internal server errors (5xx) and lack systematic capabilities for generating high-quality test cases covering both successful responses (2xx) and client-side error scenarios (4xx). They further suffer from low test reusability, complex assertion design, and poor CI/CD integration. This paper introduces RESTifAI, an LLM-powered automated testing workflow that deeply parses OpenAPI specifications to collaboratively generate reusable, semantically rich positive and negative test cases with domain-aware assertions. Its key innovation lies in synergistically integrating LLM reasoning with domain knowledge—including business logic and HTTP semantics—to enable end-to-end test generation and validation. Empirical evaluation shows that RESTifAI achieves test coverage and effectiveness comparable to AutoRestTest and LogiAgent, while industrial deployment across microservices confirms its engineering practicality and seamless CI/CD compatibility.

Technology Category

• Knowledge Representation and Reasoning: Automated Reasoning and Theorem Proving
• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information

Abstract

With this paper, we introduce RESTifAI, an LLM-driven approach for generating reusable, CI/CD ready REST API tests, following the happy-path approach. Unlike existing tools that often focus primarily on internal server errors, RESTifAI systematically constructs valid test scenarios (happy paths) and derives negative cases to verify both intended functionality (2xx responses) and robustness against invalid inputs or business-rule violations (4xx responses). The results indicate that RESTifAI performs on par with the latest LLM tools, i.e., AutoRestTest and LogiAgent, while addressing limitations related to reusability, oracle complexity, and integration. To support this, we provide common comparative results and demonstrate the tool's applicability in industrial services. For tool demonstration, please refer to https://www.youtube.com/watch?v=2vtQo0T0Lo4. RESTifAI is publicly available at https://github.com/casablancahotelsoftware/RESTifAI.

Problem

• Generates reusable REST API tests for CI/CD pipelines.
• Constructs valid and negative test scenarios for functionality.
• Addresses limitations in reusability, oracle complexity, and integration.

Innovation

• LLM-driven workflow for reusable REST API tests
• Generates happy-path and negative test scenarios systematically
• Addresses reusability, oracle complexity, and integration limitations