LumiTex: Towards High-Fidelity PBR Texture Generation with Illumination Context

2025-11-24

AI Summary

Existing PBR texture generation methods suffer from two key limitations: inaccurate material decomposition under limited lighting cues and viewpoint-inconsistent texture completion. This paper proposes an end-to-end illumination-aware material synthesis framework. Our approach jointly optimizes physically based rendering (PBR) constraints and introduces three core innovations: (1) a multi-branch generative architecture that disentangles albedo, metallic, and roughness maps under a shared illumination prior; (2) an illumination-context-aware material attention mechanism to strengthen joint illumination–material modeling; and (3) a geometry-guided UV-space inpainting module ensuring multi-view texture coherence. Extensive experiments demonstrate that our method achieves state-of-the-art performance in material decomposition accuracy, texture fidelity, and UV-space visual consistency—outperforming both open-source and commercial baselines.

Technology Category

• Computer Vision: Computational Photography, Image & Video Synthesis
• Natural Language Processing: Generation
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Economics, Online Markets and Human Computation: Economic ramifications for generative AI infrastructure and applications
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG

Abstract

Physically-based rendering (PBR) provides a principled standard for realistic material-lighting interactions in computer graphics. Despite recent advances in generating PBR textures, existing methods fail to address two fundamental challenges: 1) materials decomposition from image prompts under limited illumination cues, and 2) seamless and view-consistent texture completion. To this end, we propose LumiTex, an end-to-end framework that comprises three key components: (1) a multi-branch generation scheme that disentangles albedo and metallic-roughness under shared illumination priors for robust material understanding, (2) a lighting-aware material attention mechanism that injects illumination context into the decoding process for physically grounded generation of albedo, metallic, and roughness maps, and (3) a geometry-guided inpainting module based on a large view synthesis model that enriches texture coverage and ensures seamless, view-consistent UV completion. Extensive experiments demonstrate that LumiTex achieves state-of-the-art performance in texture quality, surpassing both existing open-source and commercial methods.

Problem

• Resolves material decomposition under limited illumination cues
• Achieves seamless and view-consistent texture completion
• Generates high-fidelity PBR textures with illumination context

Innovation

• Multi-branch generation disentangles albedo and metallic-roughness
• Lighting-aware attention injects illumination context into decoding
• Geometry-guided inpainting ensures seamless view-consistent UV completion