diversity maintenance

This study investigates how behavioral diversity influences team performance in multi-agent reinforcement learning (MARL), particularly under computational constraints and sparse reward settings, to enhance collaboration efficiency and robustness. We propose a trajectory-embedding-based behavioral distance metric, a diversity-regularized objective, heterogeneous policy initialization, and a curriculum-style perturbation training framework. Our work is the first to systematically demonstrate that behavioral heterogeneity spontaneously induces unbiased role specialization, strengthens morphological synergy, accelerates cooperative policy discovery under sparse rewards, and enables implicit skill retention and transfer. Experiments across diverse collaborative tasks show that heterogeneous teams achieve 23–41% higher average task success rates than homogeneous baselines, recover from environmental perturbations 2.8× faster, and more stably acquire reusable collaborative sub-policies.

Unsupervised skill discovery faces a fundamental trade-off between behavioral quality and diversity—particularly challenging in high-dimensional robotic control domains with rich latent skill spaces. To address this, we propose Contrastive Multi-objective Skill Discovery (ComSD), the first framework to jointly optimize a contrastive learning–driven diversity reward and a particle-filter–based exploration reward, thereby establishing a dynamic multi-objective reward mechanism in a reward-free setting. ComSD integrates adaptive weight balancing and unsupervised representation learning to unify high-fidelity skill discrimination with progressive discovery of novel behaviors. Evaluated on 32 downstream tasks, ComSD achieves state-of-the-art performance, significantly enhancing both behavioral diversity and practical utility of multi-joint robots across hierarchical exploration regimes.

Existing multi-agent reinforcement learning (MARL) methods predominantly focus on individual policy optimization while neglecting inter-agent interaction, leading to policy homogenization and poor adaptability. To address this, we propose CoDiCon—a novel MARL framework that, for the first time, integrates the sociological concept of constructive conflict into MARL. CoDiCon introduces a centralized intrinsic reward mechanism grounded in rank-based features, implemented via a parameterized reward module and a bilevel optimization architecture to dynamically balance competition and cooperation: competitive intrinsic rewards stimulate policy exchange and enhance diversity, while centralized rank-based features ensure collaborative consistency. Evaluated on the SMAC and Google Research Football (GRF) benchmarks, CoDiCon significantly outperforms state-of-the-art methods, demonstrating synergistic improvements in policy diversity, team adaptability, and task performance.

This work addresses the collapse of generation diversity in reinforcement fine-tuning, where optimization dynamics often drive model outputs toward a single solution (i.e., a Dirac delta distribution) due to misalignment between the objective function and the optimization landscape. To mitigate this, we propose DRIFT, the first framework to systematically incorporate diversity incentives into reinforcement fine-tuning. DRIFT synergistically preserves both task alignment and output diversity during policy updates through reward-concentrated subset sampling, stochastic prompt augmentation, and potential-based reward shaping. Experimental results demonstrate that DRIFT achieves Pareto superiority: it improves generation diversity by 9.08%–43.46% while maintaining equivalent task alignment, or enhances task alignment by 59.65%–65.86% under comparable diversity levels.

To address policy homogenization induced by parameter sharing in multi-agent reinforcement learning—particularly its inability to accommodate heterogeneous agent identities and task requirements—this paper proposes a zero-overhead, identity-driven adaptive subnet partitioning mechanism. Inspired by neural functional parcellation, the method employs a learnable identity encoder to generate agent-specific binary masks, which dynamically route inputs to localized subnetworks within a shared backbone, thereby enabling differentiated policy representations. Crucially, it introduces no additional parameters and is fully compatible with standard on-policy algorithms such as PPO and A2C. Extensive experiments on StarCraft II, the Multi-Agent Particle Environment (MPE), and custom heterogeneous benchmarks demonstrate an average 12.7% improvement in win rate and a 3.2× increase in inter-agent policy diversity, significantly outperforming both conventional parameter sharing and Hypernetwork-based baselines.

Traditional reinforcement learning relies on deterministic policies, which struggle to meet the demand for behavioral diversity in tasks such as language model fine-tuning or scientific discovery. This work proposes a novel paradigm based on distributions over reward functions, introducing nonlinear objectives over action sets and leveraging a principled gradient estimator derived from contextual bandits. The approach enables controllable induction of policy diversity without compromising expected return. By unifying classical policy gradient methods with action-set optimization frameworks, the proposed method demonstrates robust generation of diverse behaviors in complex tasks, significantly outperforming conventional approaches in empirical evaluations.

This work addresses the limitations of single-policy exploration in large-scale reinforcement learning and the instability or inefficiency often caused by existing ensemble methods due to excessive exploration. Through theoretical analysis, the study reveals the critical role of policy diversity in learning efficiency and proposes a coupled policy optimization approach. By introducing a KL divergence constraint within an ensemble policy gradient framework, the method explicitly regulates inter-policy diversity to enable efficient and stable structured exploration. Evaluated in large-scale parallel reinforcement learning systems, the proposed approach significantly outperforms baseline methods—including SAPG, PBT, and PPO—on complex tasks such as dexterous manipulation, achieving substantial improvements in both sample efficiency and final performance.

This study addresses a key limitation in existing reinforcement learning approaches, which often overlook individual differences when modeling biological behavior, focusing instead on optimal policies or population averages. To overcome this constraint, the work introduces a biologically interpretable framework that integrates methods from multiple subfields of reinforcement learning to construct a computational model capable of generating diverse individual behaviors. By systematically synthesizing technical strategies that support behavioral diversity, the research establishes a novel paradigm for modeling individual variation in biological agents. This paradigm effectively narrows the gap between simulated and real-world biological behaviors, offering both a theoretical foundation and practical guidance for future research in biologically plausible behavior modeling.

Existing multi-agent reinforcement learning (MARL) approaches typically bind behaviors statically to agent identities, limiting their ability to adapt dynamically to changing tasks. This work proposes a novel event-triggered paradigm that decouples agent identity from behavior, constructing a continuous behavior manifold and employing an event-driven hypernetwork to dynamically generate LoRA modules for on-the-fly reconstruction of a shared policy. To balance behavioral diversity with reward maximization, the method introduces a Neural Manifold Diversity (NMD) metric. Experimental results demonstrate that the proposed approach outperforms current state-of-the-art methods across multiple benchmarks, exhibits strong zero-shot generalization capabilities, and is the first to successfully solve tasks requiring sequential reallocation of behaviors.

Open-domain generation tasks suffer from the absence of scalar rewards, high costs of human annotation, and the tendency of existing reinforcement learning approaches to collapse output diversity. To address these challenges, this work proposes PPR-GDE, a novel method that dispenses with traditional scalar rewards and instead constructs a reinforcement learning framework based on pairwise preferences. It mitigates annotator position bias through sequential swapping and introduces, for the first time, a group-level semantic diversity metric as the reward signal, unified under a group-relative policy optimization objective. Experimental results demonstrate that PPR-GDE significantly outperforms strong baselines in role-playing tasks, achieving simultaneous improvements in both generation quality and expressive diversity. These findings validate the efficacy of pairwise preferences for subjective alignment and the contribution of the group diversity mechanism to enhanced semantic coverage.