Seg3DParts: Segmentation-Grounded Controllable Part-Level 3D Generation

2026-09-29

AI Summary

This study addresses the challenges of occlusion-induced structural ambiguity and insufficient inter-part coherence in single-image part-level 3D generation. To overcome these limitations, this work proposes a controllable 3D mesh generation framework that leverages segmentation as an explicit anchor. Methodologically, it pioneers the use of segmentation as an explicit localization signal for generation and introduces global context exchange alongside structured cross-part attention mechanisms. Combined with a newly constructed large-scale dataset, PartObjectNet, the proposed approach achieves high-quality part assembly without requiring post-processing alignment. Experimental results demonstrate that this method significantly outperforms existing approaches in geometric quality, part coherence, and controllable decomposition.

Abstract

Part-level 3D assets are essential for editing, reassembly, and interaction, yet recovering such structure from a single image remains challenging due to occlusion, ambiguous boundaries, and the need for coherent multi-part reasoning. Existing approaches struggle to achieve both controllable part-level generation and coherent multi-part structure, as part identity and spatial allocation are typically inferred implicitly. We present Seg3DParts, a segmentation-grounded framework for controllable part-level 3D generation from a single image. By treating segmentation as an explicit grounding signal, our method defines part identity during generation, enabling each component to be anchored to a corresponding image region. To ensure coherent assemblies, we introduce structured cross-part interaction that allows components to exchange global context throughout the generative process. As a result, Seg3DParts directly generates well-aligned part meshes in a shared canonical space without post-hoc alignment, supporting flexible and controllable decomposition. We further introduce PartObjectNet, a large-scale dataset with over 200K objects and 1M annotated parts. Experiments demonstrate that Seg3DParts achieves superior geometry quality, cross-part coherence, and part-level controllability over existing methods.

Problem

• Part-level 3D generation
• Single image
• Controllable generation
• Multi-part coherence
• Segmentation grounding

Innovation

• Part-level 3D Generation
• Segmentation-grounded
• Cross-part Interaction
• Controllable Decomposition
• PartObjectNet