OmniFabric: Coherent UV Space Texture Synthesis for 3D Garment Reconstruction

2026-09-24

AI Summary

This study addresses the baked-in illumination and structural inconsistencies prevalent in single-image 3D garment texture generation, which hinder physical simulation and relighting. We propose synthesizing globally consistent textures within the 2D pattern space. Our approach leverages vision-language models for initialization and employs a diffusion Transformer conditioned on 3D positional features to refine textures in the UV domain, effectively eliminating artifacts. Furthermore, we design an automated synthetic data engine to train the conditional diffusion model, enabling standardized texture extraction free from illumination contamination. The proposed method significantly outperforms existing baselines, generating high-fidelity, physically simulatable 3D garment assets suitable for downstream applications.

Abstract

Automated generation of production-ready 3D garment assets from a single image is a central challenge in digital content creation. While recent generative models have significantly advanced 3D geometry reconstruction, synthesizing high-quality textures remains a bottleneck. Existing methods often bake environmental illumination and shadows directly into the texture map, or they fail to maintain global structural coherence, making the resulting assets unusable for physical simulation and relighting. In this work, we introduce OmniFabric, a novel approach that synthesizes globally coherent texture maps directly within the 2D sewing pattern space. Given a single reference image, our pipeline utilizes an estimated 3D mesh and generative priors of powerful Vision-Language Models (VLM) to establish a complete but coarse texture initialization across the unwrapped sewing patterns. We then leverage a specialized diffusion transformer, trained via an automated synthetic data engine and conditioned on 3D positional features, to refine this initialization directly in the canonical UV domain. This effectively removes distortion and baked-in artifacts to extract a clean and normalized texture map that preserves the original garment design. Extensive experiments demonstrate that OmniFabric significantly outperforms state-of-the-art baselines, yielding photorealistic 3D garments with high-quality textures.

Problem

• 3D garment reconstruction
• texture synthesis
• single image
• UV space
• digital content creation

Innovation

• UV Space Texture Synthesis
• 3D Garment Reconstruction
• Diffusion Transformer
• Vision-Language Models
• Sewing Pattern Space