GeoMin: Data-Efficient Semi-Supervised RLVR via Geometric Distribution Modeling

2026-06-03

AI Summary

This work addresses the challenge in semi-supervised reinforcement learning with verifiable rewards (RLVR), where existing approaches often struggle to effectively leverage unlabeled data due to reliance on coarse heuristics. The authors propose a novel method that models global structural characteristics of reasoning trajectories using only a small set of labeled examples, introducing— for the first time—a geometric distribution to capture the structural disparity between correct and incorrect trajectories. This formulation yields a reliable prior for self-reward estimation, significantly enhancing the utilization of unlabeled data. Empirically, the approach surpasses fully supervised baselines by 4.1% in performance while using merely 10% of the labeled data, demonstrating exceptional data efficiency and reasoning optimization capability.

Abstract

Reinforcement learning with verifiable rewards (RLVR) significantly advances LLM reasoning, yet it faces a dilemma: standard supervised scaling is throttled by high annotation costs, while unsupervised alternatives suffer from severe model collapse. Recent semi-supervised RLVR methods address this by using a small labeled set to guide unlabeled data, achieving a promising trade-off between training efficacy and annotation cost. However, they suffer from a severe data-efficiency bottleneck due to the reliance on coarse performance heuristics, leaving a vast majority of valuable instances underutilized. To this end, we propose GeoMin, which models global feature distributions on labeled data to decode the structural discrepancy between correct and incorrect rollouts, thereby establishing a robust prior to assess the reliability of self-reward signals and fully unleash the potential of unlabeled data. Empirically, GeoMin outperforms the strongest baselines by +4.1% and even surpasses fully supervised models with only 10% of the annotations, demonstrating remarkable data efficiency.

Problem

• semi-supervised RLVR
• data efficiency
• model collapse
• annotation cost
• unlabeled data utilization

Innovation

• semi-supervised RLVR
• geometric distribution modeling
• data efficiency
• self-reward reliability
• structural discrepancy