gradient-based multi-objective optimisation

This work addresses the challenge of efficiently and provably computing the complete Pareto-optimal policy set in multi-objective reinforcement learning. We propose Iterative Pareto Reference Optimization (IPRO), a novel framework that decomposes Pareto front approximation into a sequence of constrained single-objective optimization subproblems. IPRO is the first method of its kind to provide theoretical convergence guarantees and, at each iteration, delivers an upper bound on the distance to undiscovered Pareto-optimal solutions. It requires no prior preference information or convexity assumptions and is compatible with arbitrary single-objective solvers. Empirical evaluation on standard benchmarks demonstrates that IPRO achieves state-of-the-art performance in both hypervolume and utility-based metrics. Moreover, we validate its generalizability beyond RL—e.g., to path planning—confirming robustness across diverse multi-objective optimization tasks.

In offline multi-objective optimization (MOO), generative models struggle to effectively leverage design-label data and approximate high-quality Pareto fronts. Method: This paper proposes a flow-matching-based generative optimization framework that integrates three novel components: multi-objective weighted prediction guidance, local Pareto filtering, and neighborhood distribution evolution sampling—enabling uniform coverage of the weight space and cross-distribution knowledge transfer. Unlike single-objective guidance paradigms, our framework directly models the conditional generation flow over the Pareto front, enhancing both sampling directionality and diversity. Results: Evaluated on multiple benchmark tasks, the method significantly improves the coverage, convergence, and uniformity of the Pareto solution set. State-of-the-art performance demonstrates its effectiveness and generalizability across diverse MOO settings.

Existing multi-objective optimization research predominantly focuses on conflicting objectives and Pareto fronts, overlooking the prevalent “aligned objectives” scenario in machine learning—where objectives are non-conflicting and mutually reinforcing. This work formally defines the aligned multi-objective optimization problem and breaks from the traditional Pareto paradigm by proposing the first gradient-based optimization framework tailored to this setting. Methodologically, it introduces a dynamic weight allocation and gradient normalization fusion algorithm grounded in gradient direction alignment analysis, accompanied by theoretical convergence guarantees. Compared to naive strategies such as weighted sum, the approach achieves significantly improved optimization efficiency and stability. Empirical evaluation on multi-task learning and large language model training demonstrates synchronous performance gains across all objectives, faster convergence, enhanced robustness, and scalability to large-scale, highly correlated objective sets.

Computing the exact Pareto frontier in multi-objective Markov decision processes (MO-MDPs) is computationally challenging due to exponential policy enumeration or reliance on continuous preference sampling. Method: This paper proposes a geometry-driven algorithm that exploits the structural property that the exact Pareto frontier lies precisely on the boundary of a convex polytope in value space—whose vertices correspond to deterministic policies, and whose adjacent vertices differ in action choice at exactly one state-action pair. The algorithm reduces global optimization to local edge traversal over this polytope, integrating dynamic programming with geometric analysis and using only a single scalarized MDP solution as its primitive operation. Contribution/Results: It is the first method to construct the exact Pareto frontier in polynomial time under known models, outperforming existing approximation schemes and exponential enumeration approaches. It overcomes fundamental limitations of prior methods—namely, dependence on dense preference sampling or restrictive assumptions about deterministic policies.

This work addresses the high computational cost of traditional methods for constrained bi-objective convex optimization, which require solving each instance repeatedly. The authors propose DIPS, a novel framework that leverages large language models (LLMs) as amortized generators of Pareto fronts, enabling direct generation of ordered feasible solution sets from textual problem descriptions via end-to-end fine-tuning. The approach integrates compact discretization encoding, numerically aware token initialization, and a three-stage curriculum optimization strategy to jointly align solution structure, feasibility, and front quality. Using a 7B-parameter LLM accelerated by vLLM, DIPS achieves normalized hypervolume ratios of 95.29%–98.18% across five problem classes, with single-instance inference as fast as 0.16 seconds—significantly outperforming both general-purpose and reasoning-oriented LLM baselines.

This work addresses the inefficiency of conventional multi-objective Bayesian optimization methods under extremely limited evaluation budgets, which often waste resources attempting to approximate the entire Pareto front despite decision-makers typically requiring only a single high-quality solution. To overcome this limitation, we propose the Single-Point-oriented Multi-objective Optimization (SPMO) framework, which abandons the full-coverage paradigm and instead introduces a novel single-point optimization strategy guided by final decision needs, accompanied by theoretical convergence guarantees. SPMO employs an Expected Single-Point Improvement (ESPI) acquisition function based on Sample Average Approximation (SAA), compatible with both noisy and noise-free settings, and amenable to gradient-based optimization. Empirical results demonstrate that SPMO significantly outperforms state-of-the-art methods across multiple benchmark and real-world problems, achieving superior solution quality with enhanced computational efficiency.

This work addresses the challenge of generalizing to unseen Pareto-optimal solutions in offline multi-objective optimization, where reliance on static datasets often limits performance. The authors propose a Pareto-conditional diffusion framework that formulates the optimization problem as a diffusion sampling process conditioned on objective trade-offs, eliminating the need for explicit surrogate models. By incorporating a reweighting strategy and a reference direction guidance mechanism, the method effectively steers the sampling process toward high-quality regions of the Pareto front beyond the support of the training data distribution. Experimental results demonstrate that the proposed approach significantly outperforms existing methods on standard offline multi-objective benchmarks, achieving superior performance in terms of solution diversity, stability, and consistency.

This work addresses the lack of a general framework for constructing parallel algorithm ensembles tailored to multi-objective binary optimization problems. It proposes DACMO, a domain-agnostic coevolutionary framework that decouples domain-invariant structures from instance-specific features through neural representations and, for the first time, leverages large language models (LLMs) to automatically generate optimization operators. This approach enables operator-level design of generalizable parallel algorithms without human intervention. Evaluated across four classes of multi-objective binary optimization problems, DACMO outperforms ensembles built upon classical MOEA templates and surpasses state-of-the-art baselines—those relying on handcrafted instance generators—on two of these problem classes.

This work addresses the challenge of out-of-distribution (OOD) data in offline multi-objective optimization, which often leads surrogate models to generate unrealistic and overly extreme Pareto solutions. To mitigate this issue, the authors propose DOMOO, a novel approach that incorporates cumulative risk control to alleviate OOD effects, jointly optimizes preference and Pareto parameters through nested Pareto set learning, and employs a diversity-driven selection strategy to enhance both solution quality and uniformity. The study also introduces IGD_offline, an innovative evaluation metric tailored for offline settings that balances convergence and diversity. Extensive experiments demonstrate that DOMOO achieves the best average ranking across multiple synthetic and real-world benchmarks, significantly outperforming existing methods.

This work addresses the challenges of convergence in multi-objective optimization arising from non-differentiable objective functions and abrupt structural changes in nondominated fronts. To this end, the authors propose a nonsmooth set-based gradient ascent method that uniquely integrates an amplitude indicator with a hierarchical weighting scheme. They derive its exact gradient and prove its computational complexity is equivalent to that of hypervolume gradient computation. By employing hierarchical aggregation, the method establishes a connection to infinitesimal coding, thereby elucidating the underlying mechanics of nonsmooth optimization. The algorithm further incorporates projected finite differences, repulsion and stagnation-recovery strategies, and coordinate-projected geometric expansion techniques. Experimental results on standard two- and three-objective benchmarks, as well as curved and hyperspherical Pareto fronts, demonstrate that the proposed approach efficiently and robustly approximates the Pareto front.