adaptive gradient fusion

This work addresses the challenge of gradient conflict in multi-task learning, which often degrades performance on certain tasks. While existing dynamic weighting methods such as MGDA aim to mitigate this issue, they suffer from high computational overhead and poor scalability. To overcome these limitations, this paper formulates gradient balancing as a bilevel optimization problem for the first time and introduces a zeroth-order optimization approach to efficiently decouple model training from weight adjustment. The proposed method substantially reduces computational cost while maintaining or even improving multi-task performance across both public benchmarks and industrial-scale datasets, achieving a favorable trade-off between efficiency and effectiveness.

It remains unclear whether specialized multi-task optimizers (SMTOs) inherently outperform uniform loss weighting in complex multi-task settings, particularly under gradient conflicts and inconsistent gradient norms. Method: We conduct a large-scale empirical study comparing state-of-the-art SMTOs against gradient normalization, adaptive weighting, and strong regularization strategies. Contribution/Results: While SMTOs generally achieve superior performance, uniformly weighted losses—when combined with appropriate regularization and thorough hyperparameter tuning—attain comparable results; in several task combinations, their performance converges. Our analysis reveals that prior claims of SMTO superiority were confounded by insufficient tuning and inadequate regularization. We propose a “cognitive framework for task weight design,” arguing that the efficacy of weighting mechanisms depends critically on optimization configuration—not architectural complexity. This challenges the necessity of sophisticated weighting schemes and provides new theoretical and practical support for simplicity and robustness in multi-task learning.

This work addresses a critical limitation in existing optimization-based multi-task learning (MTL) approaches: when employed with advanced optimizers such as Muon, their performance is hindered because instantaneous gradients contribute minimally to parameter updates, thereby underutilizing the optimizer’s learning dynamics. The study reveals that gradients are systematically undervalued in such optimizers and further uncovers an inherent, implicit MTL capability within Muon itself. To bridge this gap, the authors propose the Adaptive Parameter Tuning (APT) framework, which integrates an adaptive momentum mechanism to harmonize the interaction between the optimizer and MTL objectives, alongside a lightweight direction-preserving strategy to enhance Muon’s orthogonalization capacity. Extensive experiments across four mainstream MTL benchmarks demonstrate that APT consistently and significantly improves performance, offering robust gains over existing methods.

In multi-task learning (MTL), gradient conflicts among tasks often degrade performance relative to single-task models. To address this, we propose GradOPS, a method that orthogonally projects each task’s gradient onto the subspace spanned by the gradients of all other tasks—thereby systematically eliminating gradient conflicts and enabling non-conflicting, controllable task trade-offs. We provide the first theoretical analysis establishing that gradient non-conflictness is both necessary and sufficient for achieving Pareto-optimal weighting strategies. GradOPS jointly achieves global conflict suppression and diverse Pareto-optimal solution discovery, with provable convergence guarantees. Extensive experiments across heterogeneous benchmarks demonstrate that GradOPS consistently outperforms state-of-the-art MTL methods, efficiently generating multiple high-quality Pareto-optimal solutions and supporting flexible, preference-aware task customization.

To address the inefficiency and poor generalizability of manual hyperparameter tuning—particularly for learning rates—this paper proposes a dynamic online meta-optimization framework that formulates learning rate adaptation as a discounted cumulative regret minimization problem over time. The method employs a gradient-based meta-update mechanism, enabling plug-and-play integration with any first-order optimizer (e.g., SGD, Adam) to achieve decoupled, real-time, adaptive step-size optimization. Key contributions include: (i) the first formalization of meta-optimization as discounted regret minimization; and (ii) a low-complexity variant that preserves theoretical rigor while ensuring computational efficiency and strong generalization. Experiments across diverse tasks demonstrate faster convergence, enhanced robustness to initialization and task heterogeneity, competitive performance against hand-tuned optimal schedulers, and significantly lower computational overhead compared to conventional hyperparameter search methods.

This study addresses the challenges in multi-objective prompt optimization, where text gradient methods often fail due to gradient conflicts and instruction interference. It identifies and distinguishes two distinct failure modes: gradient dilution during optimization and instruction interference during inference, thereby clarifying the design boundaries for multi-objective judge customization. Drawing on multi-task learning principles, the work proposes five information-sharing architectures that decouple interactions among losses, gradients, and the language model at different levels of abstraction. Experimental results reveal that, among ten configurations, six fail to outperform the initial prompt; gradient specificity drops by 59%; and naively merging task instructions reduces the Spearman correlation coefficient by 5.3%, collectively highlighting the inherent difficulties and limitations of multi-objective text gradient optimization.

This work addresses the inefficiency and task interference in multi-task learning that arise from neglecting the structural properties of model parameter matrices. It introduces matrix geometry into multi-objective optimization for the first time, proposing an orthogonally normalized gradient update mechanism grounded in matrix-valued steepest descent theory under the spectral–nuclear norm geometry. The method guarantees convergence to Pareto stationary points even in non-convex settings. Empirical evaluations across multiple benchmarks demonstrate substantial improvements in both optimization efficiency and multi-task performance. Theoretically, the approach achieves convergence rates of $\mathcal{O}(T^{-1/2})$ in the deterministic setting and $\mathcal{O}(T^{-1/4})$ under stochastic gradients.

This study addresses the challenge of aligning learning signals with objectives in multi-reward policy optimization by proposing the ORPG method. Specifically, ORPG constructs independent clipping objectives for each reward and formulates the policy update as the unique solution to a spherical directional compromise. When gradients are compatible, it employs cosine-dependent interpolation for fusion; when conflicts arise, it performs gradient projection according to predefined priorities, thereby achieving synergistic updates within a single policy. Experimental results demonstrate that ORPG significantly outperforms baseline methods in both helpfulness–safety alignment and mathematical reasoning tasks involving correctness–cost trade-offs, effectively improving accuracy while reducing response length.

This work addresses the instability and collapse of output diversity commonly encountered in post-training with reinforcement learning, as well as the lack of a unified design principle in existing advantage function methods. The authors propose FADE, a novel framework that systematically decouples the gradient weighting structure of the advantage function by decomposing it along the sign and difficulty axes into positive and negative gradient quality components. This decomposition reveals the dynamic trade-off between exploration and exploitation, enabling an adaptive scheduling mechanism that dynamically adjusts gradient weights to balance accuracy and diversity. Evaluated on 7B and 32B models, FADE achieves peak pass@1 performance 20k and 2k training steps earlier, respectively, and demonstrates state-of-the-art accuracy–diversity trade-offs on the LiveCodeBench and AIME benchmarks.

This work addresses the challenge of integrating augmented Lagrangian and optimistic dual methods for equality-constrained optimization by proposing an additive hybrid framework that unifies matrix-valued augmentation and optimistic correction as distinct decompositions of a common correction matrix. By adaptively selecting the optimal splitting and stepsize through local spectral weighting, the method yields, for the first time, a closed-form hybrid update rule that jointly balances primal curvature and dual memory scale within a finite number of steps. Theoretical analysis reveals the equivalence and design flexibility between the two mechanisms, while experiments demonstrate that the proposed approach significantly outperforms individual strategies on nonlinear equality-constrained problems, achieving performance close to grid-search optimality and matching state-of-the-art first-order primal-dual algorithms under moderate ill-conditioning.