On the Energy Consumption of Test Generation

2025-01-16

AI Summary

Energy consumption has been largely overlooked as a critical metric in automated software testing, particularly for test generation tools like EvoSuite. Method: This study pioneers energy-aware evaluation of EvoSuite-generated Java test suites, employing hardware-level power monitoring via Intel RAPL. We systematically measure and compare energy consumption during test generation and execution across EvoSuite’s multi-strategy framework, using industrial-grade Java benchmarks. We analyze the impact of algorithm selection and code complexity on energy usage, benchmarking against manually written tests. Contribution/Results: Results reveal up to a 3.2× energy disparity among EvoSuite algorithms; energy divergence intensifies significantly for highly complex classes. On average, EvoSuite-generated tests consume 37% less energy than human-written tests while achieving equivalent or higher coverage. This work establishes the first empirical paradigm for energy assessment in automated testing, identifies key energy-intensive phases, and provides actionable insights for green software testing optimization.

Technology Category

• Search and Optimization: Evolutionary Computation
• Machine Learning: Evaluation and Analysis
• Constraint Satisfaction and Optimization: Satisfiability Modulo Theories

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Energy management for devices in mobile Web and WoT environments
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Security and Privacy: Large-scale security measurements

Abstract

Research in the area of automated test generation has seen remarkable progress in recent years, resulting in several approaches and tools for effective and efficient generation of test cases. In particular, the EvoSuite tool has been at the forefront of this progress embodying various algorithms for automated test generation of Java programs. EvoSuite has been used to generate test cases for a wide variety of programs as well. While there are a number of empirical studies that report results on the effectiveness, in terms of code coverage and other related metrics, of the various test generation strategies and algorithms implemented in EvoSuite, there are no studies, to the best of our knowledge, on the energy consumption associated to the automated test generation. In this paper, we set out to investigate this aspect by measuring the energy consumed by EvoSuite when generating tests. We also measure the energy consumed in the execution of the test cases generated, comparing them with those manually written by developers. The results show that the different test generation algorithms consumed different amounts of energy, in particular on classes with high cyclomatic complexity. Furthermore, we also observe that manual tests tend to consume more energy as compared to automatically generated tests, without necessarily achieving higher code coverage. Our results also give insight into the methods that consume significantly higher levels of energy, indicating potential points of improvement both for EvoSuite as well as the different programs under test.

Problem

• Automated Testing
• Energy Consumption
• Code Complexity

Innovation

• Eco-friendly Testing
• EvoSuite Energy Consumption
• Test Generation Efficiency