🤖 AI Summary
To address the inefficiency, time consumption, and high resource demands of manual mask generation in visual effects (VFX) production, this paper proposes a text-prompt-driven automated video segmentation pipeline. The method integrates text-guided instance segmentation for flexible object localization, fine-grained frame-wise semantic segmentation, and temporally consistent video object tracking, deployed via lightweight containerization to ensure seamless integration with artist workflows. Key contributions include: (i) the first end-to-end incorporation of text prompting into VFX-grade video segmentation; (ii) a multi-stage collaborative architecture ensuring inter-frame segmentation stability; and (iii) structured, editable outputs—including mask sequences, confidence maps, and trajectory metadata. Experiments demonstrate that the system reduces pre-compositing preparation time by 72% on average and decreases manual intervention by 89%, significantly enhancing automation and productivity across VFX pipelines.
📝 Abstract
Visual effects (VFX) production often struggles with slow, resource-intensive mask generation. This paper presents an automated video segmentation pipeline that creates temporally consistent instance masks. It employs machine learning for: (1) flexible object detection via text prompts, (2) refined per-frame image segmentation and (3) robust video tracking to ensure temporal stability. Deployed using containerization and leveraging a structured output format, the pipeline was quickly adopted by our artists. It significantly reduces manual effort, speeds up the creation of preliminary composites, and provides comprehensive segmentation data, thereby enhancing overall VFX production efficiency.