🤖 AI Summary
This study addresses the challenges of imperfect supervision signals and dynamically evolving evaluation criteria in rule-driven decision-making by proposing a framework that integrates confidence-adaptive policy optimization with evolutionary decision knowledge. Methodologically, we design the CA-GRPO algorithm to dynamically balance outcome and process rewards, alongside an evolutionary module that distills reusable knowledge from failure cases. Experiments conducted on the Qwen3.6 backbone using a multimodal content moderation dataset demonstrate that the proposed framework significantly outperforms existing baselines on industrial-scale data. Furthermore, it exhibits exceptional robustness under shifting rule conditions, effectively overcoming the limitations inherent in fixed reward-mixing approaches.
📝 Abstract
Rule-governed contextual decision tasks require models to apply specified rules to case-specific context and evidence. Written rules can leave gaps in decision guidance and process evaluation, while reference judgments vary in their support from the rules and evidence. To address these challenges, we introduce AdaptEvo, a framework for learning under imperfect supervision that couples confidence-adaptive policy optimization with evolving decision knowledge and evaluation rubrics. Its Training module uses Confidence-Adaptive GRPO (CA-GRPO) to balance outcome and process rewards according to reference confidence. Its Evolution module synthesizes reusable decision knowledge from recurring failures across training cases and refines process rubrics to detect overlooked errors. To support empirical evaluation, we construct an industrial multimodal content moderation dataset comprising a training set and In-Period and Out-of-Period test sets, with the latter collected under changed rules. Using Qwen3.6-35B-A3B, AdaptEvo achieves 61.9% exact-label accuracy and 72.2% binary decision accuracy on In-Period, exceeding GRPO by 7.5 and 3.7 percentage points, respectively. On Out-of-Period, the policy trained with CA-GRPO retains exact-label accuracy gains over the base model across evaluated checkpoints without injected decision knowledge, while GRPO declines with continued training. CA-GRPO also outperforms the tested fixed reward mixtures on both Out-of-Period metrics.