NRF-GS: Neural Residual Fields for Expressive and Compact Gaussian Splatting

2026-09-29

AI Summary

This study addresses the limitation of 3D Gaussian Splatting, which relies on per-point spherical harmonics to model view-dependent reflections, resulting in restricted representational capacity and geometric redundancy. To overcome this, we propose NRF-GS, a hybrid representation that replaces per-Gaussian spherical harmonic bases with a shared neural residual field. By integrating diffuse features with a global scene-level MLP, the method accurately predicts high-frequency directional reflection details. This approach reduces the number of Gaussians by 50% while maintaining or improving rendering quality, significantly enhancing the reconstruction of specular and high-frequency details. Consequently, it effectively resolves the issue of representational redundancy inherent in standard 3D Gaussian Splatting frameworks.

Abstract

We revisit the role of appearance modeling in 3D Gaussian Splatting (3DGS) and show that limited expressiveness in view-dependent reflectance is a key driver of representation redundancy. In standard 3DGS, low-order spherical harmonics (SH) are used, restricting the splats' ability to model high-frequency directional effects, which is typically compensated by increasing the number of splats. We propose \emph{NRF-GS: Neural Residual Fields for Gaussian Splatting}, a hybrid representation that replaces per-splat SH-bases with a shared neural residual field. Each Gaussian encodes a compact set of appearance features and a lambertian base color, while a lightweight \emph{global scene-level MLP} predicts view-dependent residuals conditioned on viewing direction, distance, and per-splat features. This formulation enhances directional reflectance modeling by combining diffuse per-splat reflectance representations with a shared global function for high-frequency details, enabling both higher expressiveness and parameter sharing across splats. Our key insight is that by accurately capturing high-frequency directional reflectance, especially in specular regions, the GS-representation becomes more expressive, reducing the need for geometrically redundant splats. As a result, NRF-GS achieves comparable or better rendering quality while reducing the number of Gaussians by up to 50\%, and produces visibly improved specular and high-frequency details.

Problem

• 3D Gaussian Splatting
• appearance modeling
• representation redundancy
• view-dependent reflectance
• spherical harmonics

Innovation

• Gaussian Splatting
• Neural Residual Fields
• Appearance Modeling
• Representation Compactness
• Specular Reflectance