PartMat: Material-Aware 3D Part Decomposition with a Single Global Latent

2026-08-03

AI Summary

This work addresses the limitations of existing 3D part decomposition methods, which predominantly rely on functional semantics while neglecting material boundaries and suffer from linearly increasing inference costs with part count. To overcome these issues, we propose an efficient material-aware decomposition framework that encodes the geometry of multi-material parts using a single global latent variable and decodes all parts in parallel within a single forward pass. Our approach integrates a diffusion model to generate initial material assignments, employs reinforcement learning to optimize part layout and suppress overlaps, and introduces a sparse voxel flow-matching model with part-wise attention to recover fine geometric details. This method achieves, for the first time, inference complexity decoupled from the number of parts, significantly improving both material decomposition accuracy and computational efficiency while preserving high geometric fidelity.

Abstract

Part-level 3D generation has recently attracted increasing attention for producing structured and editable 3D assets. However, existing methods typically decompose objects according to functional semantics rather than the editable material boundaries (e.g., fabric, wood, metal) required in practical 3D applications such as interior design. Additionally, current methods often generate parts independently, causing computational costs to scale linearly with the part count. To address these limitations, we present PartMat, an efficient material-aware 3D part decomposition pipeline that represents multi-part geometry with a single global latent. Given a reference image and a single whole-object geometry, PartMat decomposes the object into parts that follow material boundaries. First, we propose PartVAE to learn such a unified representation and decode all material parts in a single forward pass, thereby decoupling inference cost from the number of parts. Second, with this representation, a diffusion model is trained for part generation and refined via reinforcement learning for accurate material assignment and overlap suppression. Finally, to recover fine-grained geometric details, we introduce a sparse-voxel flow-matching model with part attention for geometry post-processing. Extensive experiments demonstrate that PartMat significantly outperforms existing baselines in material-aware decomposition accuracy and achieves comparable geometric quality, while maintaining efficient inference.

Problem

• material-aware decomposition
• 3D part generation
• editable material boundaries
• computational efficiency
• structured 3D assets

Innovation

• material-aware decomposition
• global latent representation
• PartVAE
• diffusion model with reinforcement learning
• sparse-voxel flow-matching