InsertFuse: A Unified Framework for Multi-Category Reference-Guided Image Insertion

2026-08-06

AI Summary

This work addresses the challenges of inter-class interference, insufficient spatial control, and weakened reference guidance in multi-category reference-guided image insertion. To overcome these issues, the authors propose a unified insertion framework that first trains category-specific expert models and then transfers their capabilities to a single student model via Insertion-on-Policy Distillation (IOPD). The method integrates Token-Aligned Geometric Conditioning (TAGC), region-balanced flow matching, and reference-aware Classifier-Free Guidance (CFG) to achieve precise spatial alignment and cross-category consistency. Evaluated on the AnyInsertion benchmark and a newly constructed multi-category test set, the approach achieves state-of-the-art performance, significantly enhancing reference fidelity and overall generation quality.

Abstract

We present InsertFuse, a unified framework for multi-category reference-guided image insertion. Its key idea is to decouple category-specific expertise learning from cross-category capability consolidation. InsertFuse first trains specialized experts for different insertion categories and then introduces Insertion On-Policy Distillation (IOPD) to consolidate their capabilities into a single student. By querying the matched expert at states visited by the student, IOPD preserves category-specific insertion behavior while mitigating the cross-category interference caused by direct joint training. To improve spatial control, we propose Token-Aligned Geometry Conditioning (TAGC), which maps mask-derived geometric cues to the visual token grid, and Region-Balanced Flow Matching, which separately normalizes prediction errors inside and outside the insertion region to prevent background-dominated and scale-dependent supervision. We further introduce Reference CFG to isolate and strengthen the guidance induced by the visual reference under fixed scene and geometry conditions, with IOPD transferring this enhanced supervision into the unified student. Extensive experiments on the public AnyInsertion benchmark and our multi-category test set demonstrate state-of-the-art performance on most metrics, showing strong reference fidelity and generation quality across diverse insertion categories.

Problem

• reference-guided image insertion
• multi-category
• spatial control
• cross-category interference
• image generation

Innovation

• Insertion On-Policy Distillation
• Token-Aligned Geometry Conditioning
• Reference CFG
• Region-Balanced Flow Matching
• Multi-Category Image Insertion