A Memetic Algorithm based on Variational Autoencoder for Black-Box Discrete Optimization with Epistasis among Parameters

2025-04-30

AI Summary

This paper addresses the challenges of strong epistasis and high computational cost in high-dimensional black-box discrete optimization. We propose VAE-MEDA, a membrane-inspired estimation-of-distribution algorithm driven by variational autoencoders (VAEs). Methodologically, it is the first to jointly embed global search distribution modeling via VAEs and bit-flip-based local search within a unified membrane-evolutionary framework, balancing modeling fidelity and computational efficiency. Key contributions include: (1) leveraging VAE latent representations to capture high-dimensional epistatic structures, thereby improving distribution estimation accuracy; and (2) incorporating lightweight local search to accelerate convergence while avoiding the expensive sampling overhead inherent in conventional EDAs. On NK-landscape benchmarks, VAE-MEDA significantly outperforms state-of-the-art algorithms—including VAE-EDA, P³, and DSMGA-II—in both convergence speed and solution quality, demonstrating its effectiveness and scalability for high-dimensional epistatic problems.

Technology Category

• Search and Optimization: Mixed Discrete/Continuous Search
• Constraint Satisfaction and Optimization: Mixed Discrete/Continuous Optimization
• Machine Learning: Evolutionary Learning

Application Category

• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Black-box discrete optimization (BB-DO) problems arise in many real-world applications, such as neural architecture search and mathematical model estimation. A key challenge in BB-DO is epistasis among parameters where multiple variables must be modified simultaneously to effectively improve the objective function. Estimation of Distribution Algorithms (EDAs) provide a powerful framework for tackling BB-DO problems. In particular, an EDA leveraging a Variational Autoencoder (VAE) has demonstrated strong performance on relatively low-dimensional problems with epistasis while reducing computational cost. Meanwhile, evolutionary algorithms such as DSMGA-II and P3, which integrate bit-flip-based local search with linkage learning, have shown excellent performance on high-dimensional problems. In this study, we propose a new memetic algorithm that combines VAE-based sampling with local search. The proposed method inherits the strengths of both VAE-based EDAs and local search-based approaches: it effectively handles high-dimensional problems with epistasis among parameters without incurring excessive computational overhead. Experiments on NK landscapes -- a challenging benchmark for BB-DO involving epistasis among parameters -- demonstrate that our method outperforms state-of-the-art VAE-based EDA methods, as well as leading approaches such as P3 and DSMGA-II.

Problem

• Black-box discrete optimization with epistasis among parameters
• Handling high-dimensional problems without excessive computational cost
• Combining VAE-based sampling with local search for improved performance

Innovation

• Combines VAE-based sampling with local search
• Handles high-dimensional epistasis efficiently
• Outperforms state-of-the-art EDA and evolutionary methods