Crayotter: Learning Long-Horizon Video Editing Agents via Group-Relative Preference Backpropagation

2026-08-03

AI Summary

This work addresses the challenge of long-horizon video editing, where feedback is often subjective, delayed, and uncalibrated, thereby failing to provide effective learning signals. The authors reformulate the editing objective as ordinal preference comparisons among candidate solutions under fixed conditions and introduce Group Relative Preference Backpropagation (GRPB). This method enables stable and efficient credit assignment by distributing zero-sum advantages to semantically meaningful editing segments, incorporating lagged policy updates and a safeguarded signal propagation mechanism. A 9B-parameter model, Crayotter, trained within this framework substantially outperforms multiple proprietary systems on AgenticVBench. Human blind evaluations and multi-benchmark assessments confirm significant improvements in both editing behavior and final output quality.

Abstract

Long-horizon video editing agents receive final-product feedback only after many interdependent decisions. Yet editing quality is subjective, admits multiple valid solutions, and is not meaningfully calibrated across heterogeneous requests, making a global scalar objective both ambiguous and temporally uninformative. Our key observation is that fixing the request, materials, and production constraints converts this subjective objective into an ordinal comparison among directly comparable alternatives. We introduce Group-Relative Preference Backpropagation (GRPB), which transforms same-task rankings into zero-sum advantages and redistributes them as bounded credit over semantic editing segments. A lagged allocator and guarded transmission prevent current judgments or unreliable estimates from directly shaping the same rollout group. We manually construct a project-disjoint, horizon-stratified suite of realistic editing tasks for training and controlled evaluation. Across matched baselines, credit interventions, external benchmarking, and blinded human evaluation, GRPB improves both editing behavior and rendered products. The resulting 9B Crayotter model surpasses several proprietary systems on AgenticVBench, supporting task-local preference reduction as a practical approach to learning from subjective, delayed outcomes. Code and all supporting materials are publicly available at https://github.com/idwts/Crayotter.

Problem

• long-horizon video editing
• subjective feedback
• delayed reward
• preference learning
• credit assignment

Innovation

• Group-Relative Preference Backpropagation
• long-horizon video editing
• preference-based learning
• credit assignment
• agentic video generation