🤖 AI Summary
Real-time testing in cloud service production environments risks interfering with live traffic, compromising service reliability and SLA compliance.
Method: This paper proposes an automated test planning framework that jointly models test configuration selection, deployment planning, and execution scheduling as a risk-constrained optimization problem. It integrates lightweight online traffic forecasting with dynamic risk mitigation strategies to coordinate test execution with production workload fluctuations.
Contribution/Results: Compared to manual approaches, the method significantly reduces configuration errors and service disruption risks. In real-world cloud service deployments, it decreases SLA violations induced by testing by 62% and shortens average test preparation time by 57%, while maintaining full test coverage and effectiveness. The framework provides a scalable, resource-aware automation solution for safe and efficient online testing of large-scale distributed systems.
📝 Abstract
Live testing is performed in the production environment ideally without causing unacceptable disturbance to the production traffic. Thus, test activities have to be orchestrated properly to avoid interferences with the production traffic. A test plan is the road map that specifies how the test activities need to be orchestrated. Developing a test plan includes tasks such as test configuration selection/generation, test configuration deployment planning, creating the test runs schedule, choosing strategies to mitigate the risk of interferences, etc. The manual design of a test plan is tedious and error prone. This task becomes harder especially when the systems are large and complex. In this paper we propose an approach for automating test plans generation. With this approach we aim at reducing service disruption that may be induced by the testing activities in production. We illustrate our approach with a case study and discuss its different aspects.