MaterialClusterGS: Palette-Based Material Decomposition and Physically-Based Relighting with 2D Gaussian Splatting

2026-06-08

AI Summary

Existing Gaussian inverse rendering methods estimate BRDF parameters independently for each primitive, resulting in an under-constrained material recovery problem, while palette-based approaches operate solely in RGB space and lack physical plausibility. This work proposes a global material palette-based decomposition framework that maps pixels to a shared set of BRDF prototypes via a continuous spatial material field. By jointly optimizing the material field, the BRDF palette, and environmental illumination, our method achieves physically plausible relighting, efficient material editing, and cross-region material transfer, while recovering a compact and spatially coherent material representation.

Abstract

We present MaterialClusterGS, a palette-based material decomposition framework for 2D Gaussian Splatting that enables physically based relighting and material editing. Existing Gaussian inverse rendering methods typically assign independent BRDF parameters to individual primitives. While flexible, this local fitting strategy makes material recovery highly under-constrained: shadows, indirect illumination, geometric errors, and visibility residuals can be absorbed into thousands of slightly different local material estimates. Meanwhile, recent palette-based appearance methods operate solely in RGB space without modeling physical materials or illumination. To bridge this gap, we represent scene materials using a compact global palette of shared BRDF prototypes assigned via a continuous spatial material field. Without shared material structure, editing one region does not propagate consistently to others of the same material, making per-primitive decompositions impractical for editing. We jointly optimize the material field, palette prototypes, and environment lighting under a physically based rendering objective. The resulting framework recovers compact, spatially coherent attributes directly usable for material editing, relighting, and transfer.

Problem

• material decomposition
• physically-based relighting
• Gaussian Splatting
• BRDF
• inverse rendering

Innovation

• palette-based material decomposition
• 2D Gaussian Splatting
• physically-based relighting
• shared BRDF prototypes
• spatial material field