MirrorCBO: A consensus-based optimization method in the spirit of mirror descent

2025-01-21

AI Summary

This paper addresses optimization problems with convex constraints, nonsmooth and nonconvex objectives. We propose Mirror Consensus-Based Optimization (MirrorCBO), which extends consensus-based optimization (CBO) to mirror spaces via a dual-particle system and inverse mirror mapping—enabling explicit constraint satisfaction and sparse solution selection. MirrorCBO integrates gradient-free optimization with the geometric adaptivity of mirror descent. To our knowledge, this is the first work embedding mirror descent principles into the CBO framework, supporting explicit convex constraints, manifold optimization, and ℓ₁-sparse regularization. We establish theoretical convergence guarantees under Bregman divergence, yielding an explicit exponential convergence rate. Experiments demonstrate that MirrorCBO outperforms existing CBO variants on sparse and constrained optimization tasks, while maintaining robustness across nonconvex manifolds and diverse mirror-space extensions.

Technology Category

• Constraint Satisfaction and Optimization: Constraint Optimization
• Search and Optimization: Non-convex Optimization
• Machine Learning: Optimization

Application Category

• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Systems and Infrastructure for Web, Mobile and WoT: Experiences and lessons learnt from Web-based algorithms and system deployments

Abstract

In this work we propose MirrorCBO, a consensus-based optimization (CBO) method which generalizes standard CBO in the same way that mirror descent generalizes gradient descent. For this we apply the CBO methodology to a swarm of dual particles and retain the primal particle positions by applying the inverse of the mirror map, which we parametrize as the subdifferential of a strongly convex function $phi$. In this way, we combine the advantages of a derivative-free non-convex optimization algorithm with those of mirror descent. As a special case, the method extends CBO to optimization problems with convex constraints. Assuming bounds on the Bregman distance associated to $phi$, we provide asymptotic convergence results for MirrorCBO with explicit exponential rate. Another key contribution is an exploratory numerical study of this new algorithm across different application settings, focusing on (i) sparsity-inducing optimization, and (ii) constrained optimization, demonstrating the competitive performance of MirrorCBO. We observe empirically that the method can also be used for optimization on (non-convex) submanifolds of Euclidean space, can be adapted to mirrored versions of other recent CBO variants, and that it inherits from mirror descent the capability to select desirable minimizers, like sparse ones. We also include an overview of recent CBO approaches for constrained optimization and compare their performance to MirrorCBO.

Problem

• Convex Optimization
• Derivative-Free Optimization
• Sparse Optimization

Innovation

• MirrorCBO
• ConvexConstraintsOptimization
• MirrorDescentEnhancement