Genetic Programming for Explainable Manifold Learning

2024-03-21

arXiv.org

AI Summary

Existing manifold learning methods lack explicit, interpretable functional mappings, limiting their applicability in decision-critical, transparent scenarios; meanwhile, current genetic programming (GP)-based approaches, though capable of generating mappings, often yield structurally redundant and semantically opaque expressions. This paper proposes GP-EMaL—the first interpretable manifold learning framework supporting customizable complexity regularization. Its core innovation lies in embedding multi-objective constraints directly into the GP optimization process, jointly maximizing mapping fidelity and user-specified complexity metrics—including tree depth, node-type weighting, symmetry, and scale invariance. Experiments on standard benchmarks demonstrate that GP-EMaL achieves geometric fidelity comparable to state-of-the-art (SOTA) manifold learning methods, while producing significantly smaller, more concise, and semantically readable functional mapping trees. To our knowledge, GP-EMaL is the first method to simultaneously enhance interpretability and modeling quality without compromising geometric accuracy.

Technology Category

• Machine Learning: Learning with Manifolds
• Search and Optimization: Mixed Discrete/Continuous Search
• Knowledge Representation and Reasoning: Computational Complexity of Reasoning

Application Category

• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs
• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods

Abstract

Manifold learning techniques play a pivotal role in machine learning by revealing lower-dimensional embeddings within high-dimensional data, thus enhancing both the efficiency and interpretability of data analysis by transforming the data into a lower-dimensional representation. However, a notable challenge with current manifold learning methods is their lack of explicit functional mappings, crucial for explainability in many real-world applications. Genetic programming, known for its interpretable functional tree-based models, has emerged as a promising approach to address this challenge. Previous research leveraged multi-objective GP to balance manifold quality against embedding dimensionality, producing functional mappings across a range of embedding sizes. Yet, these mapping trees often became complex, hindering explainability. In response, in this paper, we introduce Genetic Programming for Explainable Manifold Learning (GP-EMaL), a novel approach that directly penalises tree complexity. Our new method is able to maintain high manifold quality while significantly enhancing explainability and also allows customisation of complexity measures, such as symmetry balancing, scaling, and node complexity, catering to diverse application needs. Our experimental analysis demonstrates that GP-EMaL is able to match the performance of the existing approach in most cases, while using simpler, smaller, and more interpretable tree structures. This advancement marks a significant step towards achieving interpretable manifold learning.

Problem

• Lack of explicit functional mappings in manifold learning
• Complex mapping trees reduce explainability in GP models
• Balancing manifold quality and tree complexity for interpretability

Innovation

• Genetic Programming for explicit functional mappings
• Penalizes tree complexity to enhance explainability
• Customizable complexity measures for diverse needs