Latent Mutants: A large-scale study on the Interplay between mutation testing and software evolution

2025-01-03

AI Summary

This study addresses the prediction of “latent faults” in software version evolution—specifically, mutants that survive in the current version but are subsequently killed in a later version (termed “latent mutants”). We formally define this concept for the first time and empirically observe an average latency of 104 days, demonstrating both observability and timeliness. Methodologically, we generate mutants using PITest and develop a random forest model integrating mutant operator types with code change features. Evaluations across 13 open-source projects comprising 131,000 mutants show that latent mutants constitute 3.5% of all mutants, with our model achieving 86% accuracy and 67% balanced accuracy. Key contributions include: (1) formal definition and empirical characterization of latent mutants; (2) discovery of systematic correlations between latent mutants and both code evolution patterns and mutation operators; and (3) the first high-accuracy, interpretable predictive framework for latent fault identification.

Technology Category

• Machine Learning: Evaluation and Analysis
• Data Mining & Knowledge Management: Anomaly/Outlier Detection
• Reasoning under Uncertainty: Relational Probabilistic Models

Application Category

• Web Mining and Content Analysis: Models for Web evolution
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

In this paper we apply mutation testing in an in-time fashion, i.e., across multiple project releases. Thus, we investigate how the mutants of the current version behave in the future versions of the programs. We study the characteristics of what we call latent mutants, i.e., the mutants that are live in one version and killed in later revisions, and explore whether they are predictable with these properties. We examine 131,308 mutants generated by Pitest on 13 open-source projects. Around 11.2% of these mutants are live, and 3.5% of them are latent, manifesting in 104 days on average. Using the mutation operators and change-related features we successfully demonstrate that these latent mutants are identifiable, predicting them with an accuracy of 86% and a balanced accuracy of 67% using a simple random forest classifier.

Problem

• Software Updates
• Latent Errors Prediction
• Error-Fix Relationship

Innovation

• Mutation Testing
• Latent Error Prediction
• Program Change Analysis