CapField-OPD: Learning Continuous Capability Fields via Joint-Anchored Multi-Teacher On-Policy Distillation for Flow Models

2026-10-01

AI Summary

This study addresses the poor robustness and lack of explicit controllability in multi-teacher knowledge distillation caused by the coupling between capabilities and prompt semantics. To overcome these limitations, this work proposes a jointly anchored online policy distillation method based on continuous capability fields. Specifically, the approach leverages flow-based models to construct an explicitly parameterized continuous capability space, where coordinate mappings are optimized via a calibration set. This design enables dynamic adjustment of capability intensities and combination weights during inference. Experimental results demonstrate that the proposed framework surpasses the performance upper bound of individual experts on compositional generation tasks, achieving highly robust and fine-grained controllable knowledge fusion.

Abstract

Reward-specialized post-training produces strong experts for flow-based generative models, while multi-teacher on-policy distillation (OPD) consolidates their capabilities into a single student. Existing methods, however, route each prompt to a single teacher according to its semantic category, implicitly binding the desired capability to prompt content. This coupling makes capability invocation vulnerable to prompt perturbations and prevents users from explicitly adjusting the strength of the desired capability at inference time. In this work, we introduce CapField-OPD, an OPD framework that integrates multiple teachers into a continuous capability field through explicit capability coordinates. We use teacher models as anchors to construct this field, with the coordinates determining how their outputs are combined. Each capability configuration thus receives a unique supervision target, and capability control no longer depends on prompt semantics. Since the training anchors may not be optimal at inference time, we further profile the learned field on a small calibration set. The coordinate with the highest mean reward serves as the recommended default, while coordinates that are frequently optimal offer a promising candidate set for test-time scaling. Extensive experiments on compositional generation, text rendering, and visual aesthetics demonstrate that CapField-OPD consolidates multiple specialized teachers into a single student while preserving or surpassing their performance, reliably invokes the desired capabilities under semantics-preserving prompt variations, and supports continuous capability control and coordinate-based test-time scaling.

Problem

• multi-teacher distillation
• flow models
• capability control
• prompt robustness
• continuous capability field

Innovation

• Continuous Capability Fields
• On-Policy Distillation
• Flow Models
• Test-Time Scaling
• Joint-Anchored