TopoMap: A Feature-based Semantic Discriminator of the Topographical Regions in the Test Input Space

2025-09-03

AI Summary

Systematically identifying input feature regions responsible for model failures remains challenging in deep learning testing. Method: This paper proposes an input-space partitioning method based on explicit topological modeling, integrating dimensionality reduction, clustering, and deep neural network–driven automated configuration evaluation. Operating in a black-box, model-agnostic setting, it constructs semantically interpretable and structurally distinguishable topological maps of the input feature space. Contribution/Results: Its core innovation lies in formulating embedding and clustering parameter selection as a learnable evaluation task, enabling end-to-end optimization. Empirical evaluation demonstrates that the generated topological regions significantly outperform random sampling in mutant killing: killing rate increases by 35% for killable mutants and by 61% for non-killable mutants. This substantially enhances test effectiveness and fault localization capability.

Technology Category

• Machine Learning: Evaluation and Analysis
• Search and Optimization: Sampling/Simulation-based Search
• Constraint Satisfaction and Optimization: Satisfiability Modulo Theories

Application Category

• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods

Abstract

Testing Deep Learning (DL)-based systems is an open challenge. Although it is relatively easy to find inputs that cause a DL model to misbehave, the grouping of inputs by features that make the DL model under test fail is largely unexplored. Existing approaches for DL testing introduce perturbations that may focus on specific failure-inducing features, while neglecting others that belong to different regions of the feature space. In this paper, we create an explicit topographical map of the input feature space. Our approach, named TopoMap, is both black-box and model-agnostic as it relies solely on features that characterise the input space. To discriminate the inputs according to the specific features they share, we first apply dimensionality reduction to obtain input embeddings, which are then subjected to clustering. Each DL model might require specific embedding computations and clustering algorithms to achieve a meaningful separation of inputs into discriminative groups. We propose a novel way to evaluate alternative configurations of embedding and clustering techniques. We used a deep neural network (DNN) as an approximation of a human evaluator who could tell whether a pair of clusters can be discriminated based on the features of the included elements. We use such a DNN to automatically select the optimal topographical map of the inputs among all those that are produced by different embedding/clustering configurations. The evaluation results show that the maps generated by TopoMap consist of distinguishable and meaningful regions. In addition, we evaluate the effectiveness of TopoMap using mutation analysis. In particular, we assess whether the clusters in our topographical map allow for an effective selection of mutation-killing inputs. Experimental results show that our approach outperforms random selection by 35% on average on killable mutants; by 61% on non-killable ones.

Problem

• Grouping test inputs by failure-inducing features in DL models
• Creating topographical maps of input feature space using embeddings
• Automatically selecting optimal clustering configurations for input discrimination

Innovation

• Topographical map creation via input feature space
• Dimensionality reduction and clustering for discriminative grouping
• DNN-based evaluation for optimal configuration selection