curriculum learning

This work addresses the growing risks of misuse and loss of control associated with the broad applicability of foundation models, which existing alignment methods struggle to mitigate through hard behavioral constraints. It establishes capability control as a core objective distinct from alignment and introduces a defense-in-depth framework spanning data, learning, and system layers to enforce multi-granular behavioral constraints throughout the model lifecycle. By integrating techniques such as data distribution shaping, representational intervention, and runtime input/output/action-level safeguards, the paper systematically constructs pathways for capability control. It further identifies critical challenges—including the dual-use nature of knowledge and combinatorial generalization—offering a new paradigm for developing safe and controllable AI systems.

This work addresses the limitations of existing curriculum learning approaches, which rely on static or computationally expensive dynamic difficulty assessments and struggle to generate efficient, learner-specific training sequences. The authors propose a novel problem difficulty evaluation mechanism based on a relative measure of model capability, introducing and formally defining “transitional problems”—critical instances that shift from difficult to easy as the model’s competence improves. Leveraging this insight, they construct an adaptive curriculum that aligns dynamically with the learner’s evolving capacity, yielding a personalized, interpretable, and computationally efficient training trajectory. Experiments on chess and mathematical reasoning tasks demonstrate that the proposed strategy significantly outperforms current methods, effectively facilitating transitions to higher levels of model performance.

Improving the performance of large language models (LLMs) on complex reasoning tasks—while maintaining efficiency and scalability—remains a central challenge in AI research. This paper introduces a difficulty-aware, multi-stage reinforcement learning (RL) paradigm, proposing the first cross-domain (mathematics + code) joint RL training framework. Our approach integrates quantitative difficulty estimation, dynamic curriculum scheduling, and multi-task hybrid reward modeling. By progressively exposing models to increasingly challenging tasks and applying hierarchical data filtering, we significantly enhance the generalization and reasoning capabilities of small-parameter models (1.5B). Specifically, our method achieves 42.3% accuracy on AIME-2024 and 89.5% on MATH-500—substantially outperforming same-scale baselines. To foster reproducibility and community advancement, we will open-source the curated dataset, establishing a new paradigm and benchmark resource for efficient, scalable reasoning research.

To address the prominent GPU memory bottleneck in large language model (LLM) pretraining, this paper proposes the Staged Parameter-Efficient Training (SPET) framework. SPET is the first to deeply integrate parameter-efficient fine-tuning techniques—such as LoRA—into the *entire* pretraining pipeline, synergistically combining gradient checkpointing with staged architectural expansion to enable dynamic model growth and on-demand memory optimization. Implemented in PyTorch, SPET introduces a memory-aware training scheduler that reduces peak GPU memory consumption by up to 53.9% versus full-parameter baselines, while preserving pretraining performance. Downstream task performance after instruction tuning remains unchanged. The core contribution lies in bridging the paradigmatic divide between standard pretraining and parameter-efficient adaptation, establishing a scalable, memory-efficient, and unified pretraining paradigm.

In large-scale pretraining, learning rate scheduling critically influences both training efficiency and model performance. This work proposes two paradigms—Fitting and Transfer. The Fitting paradigm establishes, for the first time, a scaling law for learning rate search factors, reducing hyperparameter tuning complexity from O(n³) to O(n·C_D·C_η). The Transfer paradigm extends μTransfer to Mixture-of-Experts (MoE) architectures and generalizes it across multiple hyperparameter dimensions, including depth, weight decay, and token length. Empirical results demonstrate that while μTransfer exhibits limited scalability in large-scale settings, the Fitting paradigm—grounded in the derived scaling law—offers superior scalability and practicality, providing a systematic guideline for hyperparameter tuning in industrial-scale pretraining.

This work addresses the feedback loops that arise after model deployment due to performativity—wherein the model’s predictions influence the data distribution—particularly under strong interventions where the convergence behavior of retraining remains poorly understood. The paper introduces the “stable signal principle,” positing that the prediction target contains an intrinsic component independent of the model (e.g., inherent item quality), and leverages this insight to analyze the dynamics of regularized repeated risk minimization. Theoretically, it establishes that as long as a non-zero stable signal exists, retraining converges geometrically to its direction, even when model-induced effects dominate. This reveals a novel role for regularization in mitigating performative feedback and extends the framework to nonlinear, heterogeneous, and time-varying settings—including language models—thereby explaining the observed stability of training on generated data.

This work addresses the instability caused by abrupt representation shifts when directly optimizing target loss under distribution shift, which often degrades useful features learned on the source task. To mitigate this issue, the authors propose a loss smoothing approach that interpolates between the source and target loss functions during the early phase of adaptation, enabling a gradual transition of the optimization objective. This method introduces, for the first time, a systematic mechanism for smoothing the target objective function. It consistently enhances adaptation stability and performance across diverse scenarios—including supervised distribution shift, vision model transfer, offline-to-online and online reinforcement learning, and language model fine-tuning—while effectively preserving informative source-domain features.

This work addresses the instability and performance degradation commonly observed during fine-tuning of pre-trained models, which often stems from gradient cancellation leading to optimization collapse. To mitigate this issue, the paper introduces, for the first time in the context of fine-tuning, a dynamic gradient scaling mechanism, proposing the Dynamic Scaled Gradient Descent (DSGD) algorithm. DSGD adaptively attenuates the gradient magnitudes of correctly classified samples, thereby effectively alleviating gradient cancellation. The method substantially enhances fine-tuning stability and robustness, consistently reducing performance variance and achieving higher accuracy than existing approaches across multiple benchmark datasets and large-scale models.

Existing code generation datasets often suffer from imbalanced difficulty levels, inconsistent formatting, and variable quality, which hinder model performance on complex programming tasks. This work proposes a difficulty-aware data curation paradigm, implementing a four-stage pipeline—collection, cleaning, filtering, and validation—and introduces an LLM-based predict-calibrate-select mechanism to automatically identify high-difficulty, novel competitive programming problems based on five weighted dimensions, yielding the high-quality MicroCoder dataset. When combined with reinforcement learning algorithms such as GRPO, the approach achieves a threefold improvement in training efficiency within 300 steps on LiveCodeBench compared to baseline methods, with an overall performance gain of up to 17.2%, particularly demonstrating substantial gains on medium- and high-difficulty tasks.

This work addresses the strong dataset dependency in post-training for reinforcement learning, where conventional fixed scheduling strategies struggle to dynamically balance exploration and exploitation and fail to adaptively adjust hyperparameters such as regularization. To overcome this limitation, the authors propose a tree-search framework powered by large language model (LLM) agents that automatically diagnoses trajectory pathologies and jointly optimizes multiple hyperparameters across multi-stage training. The study uncovers, for the first time, structural patterns wherein capacity-related parameters exhibit monotonic accumulation while regularization parameters oscillate, leading to transferable adaptive scheduling principles that uniformly explain the common dynamics of policy behavior across diverse tasks. Evaluated on four GRPO benchmarks, the method achieves performance gains of 9%–140% over baselines, substantially outperforming grid search (+6%–15%), random search, and skill-based agents.