Enhanced Video Text Editing with Trajectory-Aligned Glyph Rendering

2026-09-28

AI Summary

This study addresses the challenge in video text editing where diffusion models struggle to reproduce precise stroke structures, frequently resulting in garbled characters. To overcome this limitation, we propose a trajectory-aligned glyph rendering approach coupled with deep normalized feature supervision. By introducing frame-wise glyph guidance and multi-depth feature supervision via a frozen recognizer, the generation process is effectively constrained. Furthermore, we construct VTEdit, the first standardized evaluation benchmark incorporating real-world scene trajectories. Experimental results demonstrate that the proposed method significantly outperforms existing baselines in both text accuracy and background consistency, achieving a sentence-level accuracy of 0.9408 while obtaining the highest user preference scores.

Abstract

Video text editing aims to replace or add text in a video while keeping the rest of the video unchanged, which requires the edited text to be correct in every frame and to move coherently with the scene. Despite the remarkable progress of video diffusion models, they struggle to reproduce exact stroke structures and often produce garbled or wrong characters, especially for characters with complex strokes. To address this, we propose a trajectory-aligned glyph rendering reference that provides explicit per-frame glyph guidance following the position and perspective of the text, and a depth-normalized recognizer feature supervision that supervises the generated text on multi-depth features of a frozen text recognizer with per-depth normalized errors, targeting stroke errors overlooked by the diffusion loss. We further build VTEdit, a benchmark of 288 real-scene clips with 440 annotated text trajectories covering text replacement and text addition, which will be publicly released to facilitate future research. Experiments on VTEdit show that our method outperforms image text editing methods, video editing methods, and commercial models in text accuracy and background preservation, achieving a sentence accuracy of 0.9408, and receives the highest preference in a user study.

Problem

• Video Text Editing
• Character Accuracy
• Stroke Structure
• Video Diffusion Models
• Benchmark

Innovation

• Video Text Editing
• Trajectory-Aligned Glyph Rendering
• Depth-Normalized Recognizer Feature Supervision
• Video Diffusion Models
• VTEdit Benchmark