Provably Lossless Acceleration of DNN Mutation Testing via Memoization

2026-07-20

AI Summary

This work addresses the high computational cost of mutation testing for deep neural networks (DNNs), a problem exacerbated by existing acceleration techniques that rely on lossy approximations and compromise the accuracy of mutation scores. To overcome this limitation, the paper introduces Mure, a novel framework that achieves provably lossless acceleration of DNN mutation testing for the first time. Mure leverages computation graph analysis to decompose a model into a shared prefix and mutation-specific suffixes, then applies memoization to reuse intermediate results from the shared prefix across mutations. The authors formally prove that this approach is equivalent to exhaustive mutation testing. Empirical evaluation across 15 diverse DNN architectures demonstrates that Mure reduces computational overhead by 44.54% on average, maintains over 20% speedup even under 5% neuron-level mutation rates, and incurs no loss in accuracy.

Abstract

Mutation analysis has recently reemerged in the context of deep neural networks (DNNs) as a promising, but notoriously costly, approach for assessing test dataset adequacy. Existing techniques speed up DNN mutation testing through lossy approximations that trade efficiency for mutation score accuracy. This paper introduces Mure, the first provably lossless framework for accelerating DNN mutation testing via memoization. Mure is based on the idea that DNN mutants and the original model share substantial redundant computation, so during mutation testing, it executes only the mutated suffixes of each mutant and reuses the common prefix from the original model, which is computed only once. We give a formal account of memoized mutation testing, and prove that Mure is sound, i.e., it produces results equivalent to exhaustive vanilla mutation testing, and identify basic conditions under which speed-up is guaranteed. We have implemented Mure and evaluated it on 15 DNN models of various architectures, complexities, and sizes ranging from a few thousands to millions of parameters. This provides empirical evidence that Mure reduces the computational cost of mutation testing by 44.54%, on average. We also observed that while state-of-the-art techniques tend to yield higher acceleration (up to 88.97%, on average), they come at the cost of some error in mutation score. We further analyze the effect of mutation generation selection ratio on the effectiveness of Mure and observed predictable reductions in memoization opportunities with increasing the percentage of mutated neurons. We observed that Mure offers more than 20% speed-up even when as high as 5% of the neurons are mutated.

Problem

• DNN mutation testing
• lossless acceleration
• memoization
• mutation score accuracy
• computational cost

Innovation

• memoization
• mutation testing
• lossless acceleration
• deep neural networks
• computational redundancy