Flow-Map Distillation on Relation Manifolds for Image Restoration

2026-08-06

AI Summary

This work addresses a key limitation in existing knowledge distillation methods for image restoration, which typically align intermediate features or relational structures to static targets, thereby neglecting the dynamic nature of knowledge transfer. To overcome this, the authors propose modeling relational knowledge transfer as a continuous flow mapping on a relation manifold, where a learned flow operator directly predicts relational states at arbitrary time steps, enabling trajectory-level supervision. The method incorporates semigroup consistency constraints and endpoint anchoring losses to ensure global self-consistency of the mapping, effectively mitigating error accumulation and target drift. Evaluated across five image restoration tasks, the approach substantially outperforms current distillation strategies, achieving approximately 50% lower training variance and significantly improved restoration quality.

Abstract

Knowledge distillation for image restoration typically aligns intermediate features or relation matrices between teacher and student networks as static targets, ignoring the dynamic structure of the knowledge transfer process. In this paper, we propose Flow-Map Distillation on Relation Manifolds (FoRM), which reformulates relation-based knowledge transfer as a continuous flow mapping problem on the relation manifold. Rather than regressing a constant velocity field between student and teacher relation states, FoRM learns a flow map operator $\mathcal{F}_θ(\mathbf{z}, t, s)$ that directly predicts the relation state at any target time $s$ given the current state at time $t$, enabling richer trajectory-level supervision. To ensure global self-consistency of the learned flow map, we introduce a safe semigroup consistency constraint that enforces compositional agreement using ground-truth bridge states, eliminating phantom-state error accumulation. An endpoint anchoring loss further prevents the operator from drifting away from the teacher target. Extensive experiments on five image restoration tasks, including super-resolution, deraining, denoising, deblurring, and low-light enhancement, demonstrate consistent gains over state-of-the-art distillation baselines across multiple backbone architectures, reducing training variance by approximately 50\% compared to naive flow matching distillation while achieving superior restoration quality.

Problem

• knowledge distillation
• image restoration
• relation manifold
• dynamic knowledge transfer
• flow mapping

Innovation

• knowledge distillation
• relation manifold
• flow map
• image restoration
• trajectory-level supervision