RelaxFlow: Text-Driven Amodal 3D Generation

2026-03-05

AI Summary

This work addresses the challenge of image-to-3D generation under severe occlusion, where semantic ambiguity hinders accurate recovery of complete structure and object category. The study formalizes, for the first time, the task of text-driven amodal 3D generation and introduces a training-free dual-branch framework. It enforces rigid constraints on observed regions via a multi-prior consensus module while incorporating a relaxation mechanism that enables structure-level control over text prompts to plausibly complete invisible regions according to textual intent. Theoretical analysis reveals that this relaxation mechanism is equivalent to applying a low-pass filter to the generated vector field, effectively decoupling geometric structure from fine details. Evaluated on two newly curated benchmarks—ExtremeOcc-3D and AmbiSem-3D—the proposed method achieves significant improvements in both completion accuracy and visual fidelity.

Technology Category

• Computer Vision: 3D Computer Vision
• Machine Learning: Multimodal Learning
• Natural Language Processing: Generation

Application Category

• Economics, Online Markets and Human Computation: Economic ramifications for generative AI infrastructure and applications
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG

Abstract

Image-to-3D generation faces inherent semantic ambiguity under occlusion, where partial observation alone is often insufficient to determine object category. In this work, we formalize text-driven amodal 3D generation, where text prompts steer the completion of unseen regions while strictly preserving input observation. Crucially, we identify that these objectives demand distinct control granularities: rigid control for the observation versus relaxed structural control for the prompt. To this end, we propose RelaxFlow, a training-free dual-branch framework that decouples control granularity via a Multi-Prior Consensus Module and a Relaxation Mechanism. Theoretically, we prove that our relaxation is equivalent to applying a low-pass filter on the generative vector field, which suppresses high-frequency instance details to isolate geometric structure that accommodates the observation. To facilitate evaluation, we introduce two diagnostic benchmarks, ExtremeOcc-3D and AmbiSem-3D. Extensive experiments demonstrate that RelaxFlow successfully steers the generation of unseen regions to match the prompt intent without compromising visual fidelity.

Problem

• semantic ambiguity
• occlusion
• amodal 3D generation
• text-driven generation
• image-to-3D

Innovation

• text-driven 3D generation
• amodal completion
• control granularity
• relaxation mechanism
• training-free framework