Selected work

Representative Papers

CARM: Cancellation-Aware Response Masking for LLM Reinforcement Learning

Oct 01, 2026

This study addresses the issue in reinforcement learning for large language models where policy updates cause sampled responses to drift off-policy, and conventional masking obscures bidirectional drift due to the cancellation of positive and negative log-ratios. To this end, we propose Cancellation-Aware Response Masking, which introduces an absolute value mechanism that averages token-level log-ratios after taking their absolute values. This effectively prevents opposing probability shifts from canceling each other out, enabling more precise sequence-level off-policy control. We further provide theoretical proofs establishing the joint boundary conditions for accepted responses. Experimental results demonstrate that our method improves average accuracy by 3.13 percentage points on the AIME mathematical reasoning task and increases Pass@1 by 2.88 percentage points across four code benchmarks, significantly outperforming existing state-of-the-art baselines.

0 citationsRead paper

TCPO: Turn-Level Credit Policy Optimization

Aug 03, 2026

This work addresses the credit assignment problem in multi-turn reasoning, where episode-level verifier scores inadequately reflect individual action contributions to final outcomes. To resolve this, the authors propose TCPO, a method that transforms coarse episode-level rewards into fine-grained credit signals through a reference-state comparison mechanism, augmented with retrospective credit, hindsight-delayed credit, and selective counterfactual estimation. By integrating episode-level advantage construction with guidance from multiple verifier types, TCPO enables precise action-value evaluation. Empirical results demonstrate substantial improvements over existing baselines across mathematical reasoning, code generation, and AppWorld tasks, reducing the number of turns required for success and achieving state-of-the-art or tied-best Pass@8 performance on Qwen3-4B and DeepSeek-R1-Distill-Llama-8B.

0 citationsRead paper

LEAP: Lean Environment-Feedback via Adaptive Pruning for Code RL in GPU Kernel Generation

Aug 03, 2026

This work addresses the challenges of sparse rewards, high compilation latency, and signal dilution across multi-round feedback in reinforcement learning for low-level system programming tasks such as CUDA kernel generation. The authors propose LEAP, a novel framework that integrates difficulty-conditioned pruning (DCP) with a critic-free multi-round reinforcement learning paradigm. By dynamically pruning tasks to focus on high-value, complex samples and employing a ranking-based pairwise tournament reward mechanism, LEAP enables adaptive optimization without manual hyperparameter tuning. The framework combines Group Relative Policy Optimization (GRPO) with hardware-aware multi-round environment feedback, maintaining strong initial-generation performance while significantly improving robustness and convergence speed during iterative refinement—offering an efficient and practical reinforcement learning solution for low-level code generation.

0 citationsRead paper

Beyond the Best Teacher: Expanding and Compressing the Reasoning Solution Manifold

Jul 30, 2026

This work addresses the limited reasoning coverage of single reinforcement learning teacher models, which constrains their ability to comprehensively guide student models. To overcome this, the authors propose an “Expand-and-Compress” framework: first, a set of complementary teachers is constructed through iterative training, leveraging Residual Group Relative Policy Optimization (RGRPO) to broaden the solution space while preserving expert behaviors via Consensus-Residual Decomposition; subsequently, knowledge from this ensemble is efficiently distilled into a single student model using reliability-weighted Teacher-Union On-policy Distillation (TU-OPD). Experimental results demonstrate that the resulting Qwen3-1.7B student model outperforms the strongest individual teacher by 2.0%, 8.3%, and 6.9% on mathematical reasoning, code generation, and instruction-following tasks, respectively, while retaining the inference efficiency of a single model.

0 citationsRead paper

MusaCoder: Native GPU Kernel Generation with Full-Stack Training on Moore Threads GPU

Jun 03, 2026

This work addresses the challenges faced by current large language models in generating efficient and correct native GPU kernel code, as well as the instability and sparse reward issues inherent in execution-based reinforcement learning. We propose MusaCoder, a full-stack training framework tailored for CUDA and MUSA backends, which enhances code generation quality through progressive kernel synthesis, diversity-preserving rejection fine-tuning, and execution-feedback-driven reinforcement learning. Our approach introduces three key innovations: PrimeEcho, a multi-round reward mechanism; Buffered Dynamic Retry, a hard-example recovery strategy; and MirrorPop, an off-policy sequence filtering method—collectively improving training stability. Evaluated on KernelBench and a MUSA porting benchmark, our 9B model matches leading closed-source systems, while the 27B variant establishes a new state of the art, significantly outperforming both open-source and commercial baselines in correctness and empirical speedup.

0 citationsRead paper

Recent publications

Latest Papers

CARM: Cancellation-Aware Response Masking for LLM Reinforcement Learning

Oct 01, 2026

This study addresses the issue in reinforcement learning for large language models where policy updates cause sampled responses to drift off-policy, and conventional masking obscures bidirectional drift due to the cancellation of positive and negative log-ratios. To this end, we propose Cancellation-Aware Response Masking, which introduces an absolute value mechanism that averages token-level log-ratios after taking their absolute values. This effectively prevents opposing probability shifts from canceling each other out, enabling more precise sequence-level off-policy control. We further provide theoretical proofs establishing the joint boundary conditions for accepted responses. Experimental results demonstrate that our method improves average accuracy by 3.13 percentage points on the AIME mathematical reasoning task and increases Pass@1 by 2.88 percentage points across four code benchmarks, significantly outperforming existing state-of-the-art baselines.

0 citationsRead paper

TCPO: Turn-Level Credit Policy Optimization

Aug 03, 2026

This work addresses the credit assignment problem in multi-turn reasoning, where episode-level verifier scores inadequately reflect individual action contributions to final outcomes. To resolve this, the authors propose TCPO, a method that transforms coarse episode-level rewards into fine-grained credit signals through a reference-state comparison mechanism, augmented with retrospective credit, hindsight-delayed credit, and selective counterfactual estimation. By integrating episode-level advantage construction with guidance from multiple verifier types, TCPO enables precise action-value evaluation. Empirical results demonstrate substantial improvements over existing baselines across mathematical reasoning, code generation, and AppWorld tasks, reducing the number of turns required for success and achieving state-of-the-art or tied-best Pass@8 performance on Qwen3-4B and DeepSeek-R1-Distill-Llama-8B.

0 citationsRead paper

LEAP: Lean Environment-Feedback via Adaptive Pruning for Code RL in GPU Kernel Generation

Aug 03, 2026

This work addresses the challenges of sparse rewards, high compilation latency, and signal dilution across multi-round feedback in reinforcement learning for low-level system programming tasks such as CUDA kernel generation. The authors propose LEAP, a novel framework that integrates difficulty-conditioned pruning (DCP) with a critic-free multi-round reinforcement learning paradigm. By dynamically pruning tasks to focus on high-value, complex samples and employing a ranking-based pairwise tournament reward mechanism, LEAP enables adaptive optimization without manual hyperparameter tuning. The framework combines Group Relative Policy Optimization (GRPO) with hardware-aware multi-round environment feedback, maintaining strong initial-generation performance while significantly improving robustness and convergence speed during iterative refinement—offering an efficient and practical reinforcement learning solution for low-level code generation.

0 citationsRead paper

Beyond the Best Teacher: Expanding and Compressing the Reasoning Solution Manifold

Jul 30, 2026

This work addresses the limited reasoning coverage of single reinforcement learning teacher models, which constrains their ability to comprehensively guide student models. To overcome this, the authors propose an “Expand-and-Compress” framework: first, a set of complementary teachers is constructed through iterative training, leveraging Residual Group Relative Policy Optimization (RGRPO) to broaden the solution space while preserving expert behaviors via Consensus-Residual Decomposition; subsequently, knowledge from this ensemble is efficiently distilled into a single student model using reliability-weighted Teacher-Union On-policy Distillation (TU-OPD). Experimental results demonstrate that the resulting Qwen3-1.7B student model outperforms the strongest individual teacher by 2.0%, 8.3%, and 6.9% on mathematical reasoning, code generation, and instruction-following tasks, respectively, while retaining the inference efficiency of a single model.

0 citationsRead paper

MusaCoder: Native GPU Kernel Generation with Full-Stack Training on Moore Threads GPU

Jun 03, 2026

This work addresses the challenges faced by current large language models in generating efficient and correct native GPU kernel code, as well as the instability and sparse reward issues inherent in execution-based reinforcement learning. We propose MusaCoder, a full-stack training framework tailored for CUDA and MUSA backends, which enhances code generation quality through progressive kernel synthesis, diversity-preserving rejection fine-tuning, and execution-feedback-driven reinforcement learning. Our approach introduces three key innovations: PrimeEcho, a multi-round reward mechanism; Buffered Dynamic Retry, a hard-example recovery strategy; and MirrorPop, an off-policy sequence filtering method—collectively improving training stability. Evaluated on KernelBench and a MUSA porting benchmark, our 9B model matches leading closed-source systems, while the 27B variant establishes a new state of the art, significantly outperforming both open-source and commercial baselines in correctness and empirical speedup.

0 citationsRead paper