stagewise training

Existing curriculum learning methods rely on static difficulty metrics, failing to accommodate the dynamic capability evolution of large language models (LLMs) during instruction tuning—resulting in rigid, suboptimal learning trajectories. To address this, we propose CAMPUS, the first capability-aware dynamic curriculum learning framework for instruction tuning. CAMPUS continuously monitors the model’s multi-dimensional capabilities, integrates multi-perspective difficulty estimation, and introduces dynamic sub-curriculum selection alongside adaptive difficulty scheduling—thereby enabling personalized, evolution-aware instruction fine-tuning. Extensive experiments demonstrate that CAMPUS consistently outperforms state-of-the-art curriculum learning baselines on major benchmarks—including AlpacaEval and MT-Bench—with average improvements of +2.1–3.8 points—validating its effectiveness and generalizability across diverse LLMs and instruction datasets.

Curriculum Learning (CL) is a meta-learning paradigm that trains a model by feeding the data instances incrementally according to a schedule, which is based on difficulty progression. Defining meaningful difficulty assessment measures is crucial and most usually the main bottleneck for effective learning, while also in many cases the employed heuristics are only application-specific. In this work, we propose the Dual-Criterion Curriculum Learning (DCCL) framework that combines two views of assessing instance-wise difficulty: a loss-based criterion is complemented by a density-based criterion learned in the data representation space. Essentially, DCCL calibrates training-based evidence (loss) under the consideration that data sparseness amplifies the learning difficulty. As a testbed, we choose the time-series forecasting task. We evaluate our framework on multivariate time-series benchmarks under standard One-Pass and Baby-Steps training schedules. Empirical results show the interest of density-based and hybrid dual-criterion curricula over loss-only baselines and standard non-CL training in this setting.

This work proposes a reverse curriculum learning framework tailored for structurally complex and conceptually deep tasks such as advanced mathematical problem solving and code generation. The approach introduces a difficulty scoring mechanism based on structural complexity and conceptual depth, and employs a teacher–student architecture to recursively decompose challenging problems. The teacher model generates progressively simplified examples through step-by-step reasoning, thereby constructing an easy-to-hard curriculum that guides the student model in incremental learning. Experimental results on benchmarks including MATH and AIME demonstrate that this method significantly outperforms standard training strategies, effectively enhancing the model’s capacity to solve complex problems.

To address the inefficiency and poor generalizability of manual hyperparameter tuning—particularly for learning rates—this paper proposes a dynamic online meta-optimization framework that formulates learning rate adaptation as a discounted cumulative regret minimization problem over time. The method employs a gradient-based meta-update mechanism, enabling plug-and-play integration with any first-order optimizer (e.g., SGD, Adam) to achieve decoupled, real-time, adaptive step-size optimization. Key contributions include: (i) the first formalization of meta-optimization as discounted regret minimization; and (ii) a low-complexity variant that preserves theoretical rigor while ensuring computational efficiency and strong generalization. Experiments across diverse tasks demonstrate faster convergence, enhanced robustness to initialization and task heterogeneity, competitive performance against hand-tuned optimal schedulers, and significantly lower computational overhead compared to conventional hyperparameter search methods.

In multi-agent reinforcement learning (MAS) under sparse rewards, training inefficiency and policy fragility arise from delayed feedback and difficulty in sharing experience across agents. To address these challenges, this paper proposes a collaborative curriculum learning framework. Its key contributions are: (1) a multidimensional curriculum design jointly modulating task difficulty, agent count, and environmental complexity; (2) a variational evolutionary algorithm for automated subtask generation; and (3) a co-evolutionary mechanism integrating agent policy optimization with environmental model learning. The framework unifies curriculum learning, variational evolution, MAS, and environment modeling. Evaluated on five cooperative benchmarks—including MPE and Hide-and-Seek—our method achieves significant improvements over state-of-the-art approaches: 2.1× faster convergence on average and an 18.7% increase in success rate, demonstrating both effectiveness and generalizability.

This work addresses the limitations of existing curriculum learning approaches, which rely on static or computationally expensive dynamic difficulty assessments and struggle to generate efficient, learner-specific training sequences. The authors propose a novel problem difficulty evaluation mechanism based on a relative measure of model capability, introducing and formally defining “transitional problems”—critical instances that shift from difficult to easy as the model’s competence improves. Leveraging this insight, they construct an adaptive curriculum that aligns dynamically with the learner’s evolving capacity, yielding a personalized, interpretable, and computationally efficient training trajectory. Experiments on chess and mathematical reasoning tasks demonstrate that the proposed strategy significantly outperforms current methods, effectively facilitating transitions to higher levels of model performance.

This work addresses the challenge in Flow Matching models where fixed time-step sampling strategies, such as midpoint biasing, struggle to balance training efficiency and sample quality. The study reframes time-step sampling as a dynamic curriculum and reveals that the loss landscape exhibits a U-shaped difficulty distribution across time steps. To exploit this insight, the authors propose a two-stage curriculum sampling strategy: initially employing midpoint-biased sampling to accelerate structural learning, followed by a switch to uniform sampling to refine boundary details. Evaluated on CIFAR-10, the method improves the Fréchet Inception Distance (FID) from 3.85 to 3.22 and achieves peak performance within 100,000 training steps—significantly outpacing the 150,000 steps required by uniform sampling.

This work addresses the challenge of dynamically varying sample complexity in multimodal learning, which existing curriculum learning approaches struggle to accommodate as models evolve. It introduces, for the first time, Partial Information Decomposition (PID) theory into multimodal curriculum learning, proposing a progressive curriculum framework that decomposes multimodal interactions into redundant, unique, and synergistic information components. This decomposition enables a dynamic characterization of sample complexity, which in turn drives an adaptive scheduling of training samples aligned with the model’s learning progress. The resulting approach yields an interpretable, training-aware sample selection mechanism that significantly outperforms conventional training strategies and state-of-the-art baselines across multiple multimodal benchmarks, demonstrating both its effectiveness and generalizability.

In sparse-reward reinforcement learning, heterogeneous tasks exhibit divergent learning progress, rendering uniform reset curricula ineffective across all contexts. To address this challenge, this work proposes SCOUT—an online, learner-agnostic reset controller that dynamically adjusts the curriculum at the granularity of individual contexts for the first time. Leveraging only binary episode success signals, SCOUT adaptively reduces assistance upon sustained success, restores it after failure, and cautiously attempts harder starting states during stagnation. Requiring no predefined context groupings, reward function modifications, or optimizer changes, SCOUT significantly improves performance across six navigation and manipulation tasks, successfully solving three that fail entirely without auxiliary resets. Notably, under deliberately induced pacing conflicts, SCOUT concurrently learns two distinct task sets, whereas all global scheduling baselines fail.

This study addresses the challenge in high-dimensional motor skill acquisition where true skill states are unobservable and task performance poorly reflects underlying learning progress, thereby hindering effective practice design. To overcome this limitation, the authors propose an automated curriculum generation framework that integrates a human motor learning model, personalized real-time skill estimation, and stochastic nonlinear model predictive control (SNMPC), enabling model-based dynamic curriculum optimization for the first time. Evaluated in both simulation and a hand exoskeleton experiment involving 36 participants, the proposed approach significantly accelerates skill acquisition—improving learning speed by approximately 23% over random curricula and 17% over performance-heuristic baselines—demonstrating its efficacy and novelty.