Deep Reinforcement Learning-Assisted Automated Operator Portfolio for Constrained Multi-objective Optimization

2026-03-17

AI Summary

This work addresses the limited generalization capability of existing constrained multi-objective evolutionary algorithms (CMOEAs), which typically rely on fixed operators, as well as the inefficiency of current adaptive approaches that select only a single operator per generation, often leading to premature convergence and wasted evaluation budgets. To overcome these limitations, we propose CMOEA-AOP, a novel deep reinforcement learning–based method that dynamically allocates multiple optimization operators in each generation according to the current population state, thereby synergistically enhancing both convergence and diversity. The approach encodes population characteristics—including objective and constraint information—as states and uses overall improvement as the reward signal to train a deep neural network that maps states to cumulative rewards. Extensive experiments on 33 benchmark problems demonstrate that CMOEA-AOP significantly outperforms state-of-the-art CMOEAs, exhibiting superior stability and adaptability.

Technology Category

• Search and Optimization: Sampling/Simulation-based Search
• Constraint Satisfaction and Optimization: Constraint Optimization
• Machine Learning: Mixture of Experts (MoE)

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI

Abstract

Constrained multi-objective optimization problems (CMOPs) are of great significance in the context of practical applications, ranging from scientific to engineering domains. Most existing constrained multi-objective evolutionary algorithms (CMOEAs) usually employ fixed operators all the time, which exhibit poor versatility in handling various CMOPs. Therefore, some recent studies have focused on adaptively selecting the best operators for the current population states during the search process. The evolutionary algorithms proposed in these studies learn the value of each operator and recommend the operator with the highest value for the current population, resulting in only a single operator being recommended at each generation, which can potentially lead to local optima and inefficient utilization of function evaluations. To address the dilemma in operator adaptation, this paper proposes a reinforcement learning-based automated operator portfolio approach to learn an allocation scheme of operators at each generation. This approach considers the optimization-related and constraint-related features of the current population as states, the overall improvement in population convergence and diversity as rewards, and different operator portfolios as actions. By utilizing deep neural networks to establish a mapping model between the population states and the expected cumulative rewards, the proposed approach determines the optimal operator portfolio during the evolutionary process. By embedding the proposed approach into existing CMOEAs, a deep reinforcement learning-assisted automated operator portfolio based evolutionary algorithm for solving CMOPs, abbreviated as CMOEA-AOP, is developed. Empirical studies on 33 benchmark problems demonstrate that the proposed algorithm significantly enhances the performance of CMOEAs and exhibits more stable performance across different CMOPs.

Problem

• Constrained Multi-objective Optimization
• Operator Adaptation
• Evolutionary Algorithms
• Local Optima
• Function Evaluation Efficiency

Innovation

• deep reinforcement learning
• operator portfolio
• constrained multi-objective optimization
• adaptive operator selection
• evolutionary algorithm