🤖 AI Summary
Existing PBR material image generation methods suffer from limited category coverage, insufficient diversity, blurry details, and unnatural tiling artifacts. To address these challenges, we propose StableMaterials—the first semi-supervised latent diffusion framework tailored for PBR material generation. Our approach makes three key contributions: (1) a semi-supervised knowledge distillation mechanism that significantly reduces reliance on labeled data; (2) a four-step latent consistency distillation scheme coupled with tileability optimization, enabling fast inference (4 sampling steps), high-resolution outputs, and seamless, artifact-free tiling; and (3) integration of SDXL-based texture distribution alignment, adversarial distillation, and a diffusion-based refiner to enhance photorealism. Extensive experiments demonstrate that StableMaterials surpasses state-of-the-art methods across material novelty, diversity, and visual fidelity, enabling rapid, high-quality, and infinitely tileable material synthesis.
📝 Abstract
We introduce StableMaterials, a novel approach for generating photorealistic physical-based rendering (PBR) materials that integrate semi-supervised learning with Latent Diffusion Models (LDMs). Our method employs adversarial training to distill knowledge from existing large-scale image generation models, minimizing the reliance on annotated data and enhancing the diversity in generation. This distillation approach aligns the distribution of the generated materials with that of image textures from an SDXL model, enabling the generation of novel materials that are not present in the initial training dataset. Furthermore, we employ a diffusion-based refiner model to improve the visual quality of the samples and achieve high-resolution generation. Finally, we distill a latent consistency model for fast generation in just four steps and propose a new tileability technique that removes visual artifacts typically associated with fewer diffusion steps. We detail the architecture and training process of StableMaterials, the integration of semi-supervised training within existing LDM frameworks and show the advantages of our approach. Comparative evaluations with state-of-the-art methods show the effectiveness of StableMaterials, highlighting its potential applications in computer graphics and beyond. StableMaterials is publicly available at https://gvecchio.com/stablematerials.