ComplexConstraints and Beyond: Expert Rubrics for RLVR

2026-06-08

AI Summary

This work addresses the limitations of existing large language model (LLM) evaluation methods, which struggle to effectively assess complex, context-dependent instruction following and agent behaviors, often focusing only on superficial constraints. To overcome this, we propose a novel evaluation paradigm grounded in expert-designed rubrics, featuring atomic, intent-aware scoring criteria calibrated via LLM-based judges to enable precise assessment and efficient training on complex tasks. We introduce five principles for high-quality rubric design and, for the first time, leverage expert rubrics simultaneously as both evaluation instruments and reinforcement learning signals. Experiments demonstrate that models trained on our ComplexConstraints dataset exhibit substantial improvements—15.5% and 12.2% gains in instruction-following performance for 4B and 235B parameter models, respectively—and show strong generalization to unseen enterprise-level tasks, with notable improvements on BFCL (+4.5%), Tau2-Bench (+7.4%), and Tool-Decathlon (+6.8%).

Abstract

As LLM capabilities advance rapidly, the evaluation methods used to assess them increasingly lag behind. Traditional benchmarks relied on programmatic verification of narrow, surface-level constraints, but real-world instruction following and agentic tasks demand assessment of nuanced, context-dependent behaviors that resist simple scripted checks. We present a systematic analysis of expert-curated rubric-based evaluation as an alternative paradigm, drawing on empirical evidence from two domains: complex instruction following and enterprise agentic tasks. We first articulate five design principles for constructing high-quality rubrics, including Maximum Viable Atomicity, intent-aware criterion design, and iterative LLM-judge calibration. To validate these principles, we introduce ComplexConstraints, a new expert-curated instruction-following dataset in which each prompt is paired with 10-40 atomic rubric criteria. We demonstrate that these expert rubrics are not only better evaluation instruments but also highly effective training signals: training on approximately 1,000 ComplexConstraints examples yields +15.5% improvement for a 4B-parameter model and +12.2% for a 235B-parameter model on instruction following, while single-epoch RL training on a rubric-graded enterprise environment produces gains that transfer to out-of-distribution benchmarks the model was never trained on (+4.5% BFCL, +7.4% Tau2-Bench, +6.8% Tool-Decathlon). Our findings establish that expert-authored rubrics improve both the measurement and the development of frontier LLM capabilities, serving as effective evaluation and RL training signals.

Problem

• LLM evaluation
• instruction following
• agentic tasks
• rubric-based assessment
• complex constraints

Innovation

• expert rubrics
• ComplexConstraints
• atomic evaluation criteria
• RL training signals
• instruction following