ALIVE: Interaction-Aligned Object Insertion for First-Frame-Guided Video Editing

2026-10-06

AI Summary

This study addresses the incoherent interactions between inserted objects and scenes in video editing by proposing a first-frame-guided interaction alignment framework. The method leverages 3D rendering to construct 35,000 pairs of high-quality edited data and introduces a vision-language model (VLM) to predict interaction guidance signals, ensuring coordinated object motions and consistent backgrounds. Additionally, a unified VLM-based evaluation protocol and a novel benchmark are designed. Experimental results demonstrate that, without VLM guidance, the overall score improves by 43.9% over the baseline, while incorporating the VLM further increases the interaction score by 0.95 points. These findings indicate significant enhancements in both the realism and interaction quality of video editing.

Abstract

Current video editors can insert objects but often struggle to make them participate in interactions such as being picked up or manipulated. We introduce ALIVE, a framework that makes inserted objects "alive" through coherent interactions with the source video's contents, using an edited first frame and an instruction naming only the added object. We curate 35,800 editing pairs combining 3D-rendered, model-generated, and real-world videos with general editing pairs from ROSE. Each pair differs in the target object's presence while preserving the surrounding action, teaching editors coordinated object behavior and source preservation. We further train a vision-language model (VLM) to predict interaction guidance from the same inputs. We introduce the ALIVE-interaction benchmark to assess interaction fidelity, source preservation, and visual coherence using a unified VLM-based protocol, and evaluate on the general video object insertion benchmark. Without VLM guidance, ALIVE improves Overall over the strongest evaluated baseline by 43.9% and 4.4% on the two benchmarks, respectively. VLM-predicted guidance further improves the ALIVE-interaction score by 0.95 points without additional user inputs.

Problem

• video editing
• object insertion
• interaction fidelity
• source preservation
• visual coherence

Innovation

• Video Editing
• Object Insertion
• Interaction Alignment
• Vision-Language Model
• First-Frame Guidance