Automated Test-Case Generation for REST APIs Using Model Inference Search Heuristic

2024-12-04

arXiv.org

AI Summary

To address insufficient system-level coverage in evolutionary testing of microservice REST APIs, this paper proposes MISH: a search heuristic grounded in online, real-time automata learning (an L*-variant). MISH dynamically infers system-level behavioral paths by parsing multi-service log streams and uses these inferred models to define test case fitness—thereby overcoming the limitations of conventional, unit-level heuristics based on code branch distances. This work is the first to integrate online automata learning into the search process of API testing, enabling systematic abstraction of system-level behavior, path-based fitness evaluation, and seamless integration with the EvoMaster framework. Evaluated on six real-world microservice benchmarks, MISH improves system-level behavioral coverage by 23.6% over MOSA and significantly enhances the capability to trigger cross-service anomalies.

Technology Category

• Machine Learning: Evolutionary Learning
• Search and Optimization: Evolutionary Computation
• Multiagent Systems: Multiagent Learning

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics

Abstract

The rising popularity of the microservice architectural style has led to a growing demand for automated testing approaches tailored to these systems. EvoMaster is a state-of-the-art tool that uses Evolutionary Algorithms (EAs) to automatically generate test cases for microservices' REST APIs. One limitation of these EAs is the use of unit-level search heuristics, such as branch distances, which focus on fine-grained code coverage and may not effectively capture the complex, interconnected behaviors characteristic of system-level testing. To address this limitation, we propose a new search heuristic (MISH) that uses real-time automaton learning to guide the test case generation process. We capture the sequential call patterns exhibited by a test case by learning an automaton from the stream of log events outputted by different microservices within the same system. Therefore, MISH learns a representation of the systemwide behavior, allowing us to define the fitness of a test case based on the path it traverses within the inferred automaton. We empirically evaluate MISH's effectiveness on six real-world benchmark microservice applications and compare it against a state-of-the-art technique, MOSA, for testing REST APIs. Our evaluation shows promising results for using MISH to guide the automated test case generation within EvoMaster.

Problem

• Microservices Testing
• Evolutionary Algorithms
• Coverage Shortage

Innovation

• Real-time Learning Automata
• Microservice Interface Testing
• EvoMaster Enhancement