AlbedoEdit: Unified Instance-Level Video Editing with Albedo Guidance

2026-05-31

AI Summary

Existing video editing methods struggle to uniformly support instance-level fine-grained manipulations—such as object insertion, removal, and texture modification—and often face a trade-off between flexibility and photorealism. This work proposes a unified albedo-guided video editing framework that, for the first time, leverages illumination-invariant albedo maps as the user-editing interface. By fine-tuning a foundational video model conditioned on albedo maps and trained on a newly constructed synthetic paired dataset, the method enables end-to-end RGB video editing. It implicitly models higher-order lighting effects, including specular reflections and soft shadows, achieving superior performance across diverse editing tasks. Both qualitative and quantitative evaluations demonstrate significant improvements in editing photorealism and temporal consistency over current state-of-the-art approaches.

Abstract

Video generative models have achieved remarkable progress in synthesizing photorealistic video sequences. However, enabling broader and more creative downstream applications requires fine-grained instance-level video editing, including object insertion, object removal, and texture editing, which has emerged as a prominent yet challenging problem. Existing approaches either propose unified generative frameworks with only coarse semantic control, or design task-specific frameworks for individual editing tasks, limiting their flexibility and applicability across diverse real-world scenarios. To address these limitations, we propose AlbedoEdit, a unified generative video editing framework that jointly supports object insertion, object removal, and texture editing. Our key insight is that the intrinsic albedo map, which is invariant to lighting and contains no specularity, shadowing and inter-reflection effects, provides an effective and user-friendly mechanism for specifying fine-grained appearance edits. Built upon video foundation models, AlbedoEdit is fine-tuned to translate source RGB videos into edited RGB videos, conditioned on a user-edited first-frame albedo. Trained on a new paired synthetic dataset covering all three editing tasks, AlbedoEdit implicitly learns to harmonize edited contents and simulate complex real-world visual effects triggered by editing operations, including specular highlights, soft shadows, and mirror reflections. AlbedoEdit demonstrates superior performance over state-of-the-art video editing approaches, both qualitatively and quantitatively. Project webpage is https://vcai.mpi-inf.mpg.de/projects/AlbedoEdit/.

Problem

• instance-level video editing
• object insertion
• object removal
• texture editing
• video generative models

Innovation

• Albedo Guidance
• Unified Video Editing
• Instance-Level Editing
• Video Foundation Models
• Appearance Harmonization