Fashion-3DLR: A Controllable 3D Garment Generation Using Pairwise Fashion Elements for Intelligent Design

2026-07-25

AI Summary

Existing 3D garment generation methods struggle to simultaneously achieve structural plausibility and diversity, and often lack semantic modeling of complex fashion elements. This work proposes Fashion-3DLR, a novel framework that, for the first time, incorporates paired 2D fashion cues—such as sketches and textures—into 3D garment synthesis. Central to this approach is the Garment Feature Fusion Diffusion Transformer (GFF-DiT), a module designed to bridge the semantic gap between 2D fashion representations and 3D geometry by generating geometric latent variables in a learned latent space. These latents can be decoded into either 3D Gaussians or meshes, enabling physical simulation and virtual try-on even for non-watertight garments. The method significantly outperforms existing approaches in terms of structural fidelity, diversity, and applicability to downstream tasks.

Abstract

AI-generated content (AIGC) has made significant progress, with 2D generative models becoming ready-to-use tools for the digital fashion industry. However, 3D garment generation remains in its nascent stage, where in the realm of fashion, the semantic information of diverse design elements exhibits intricate coupling relationships in 3D representations, posing substantial challenges for generating diverse 3D garments. In this work, to handle the above problem, We introduce Fashion-3DLR, a novel 3D garment generation framework that utilizes diverse design elements to create high-quality, versatile 3D garment assets. Specifically, to bridge the semantic gaps between different fashion elements, we propose a Garment Feature Fusion Diffusion Transformer (GFF-DiT) module to integrate 2D fashion design elements, e.g., sketch and texture, into latent space. Within the latent space, we then employ a rectified flow transformer to generate geometry latents, which can be decoded into various 3D garment representations, including 3D Gaussians and meshes. Furthermore, we integrate Fashion-3DLR into downstream tasks, achieving the 3D Gaussian Splatting (3DGS)-driven cloth physical simulation and mesh-based virtual try-on. Experimental results indicate that Fashion-3DLR surpass the previous state-of-the-art methods, which verify that the proposed work can generate well-structured, non-watertight garments capable of physical simulation and virtual try-on, underscoring its potential as a versatile 3D garment design tool.

Problem

• 3D garment generation
• fashion elements
• semantic coupling
• diverse 3D garments
• digital fashion

Innovation

• 3D garment generation
• diffusion transformer
• latent space fusion
• rectified flow
• virtual try-on