Neural Acquisition & Representation of Subsurface Scattering

2026-06-01

AI Summary

This work proposes a high-accuracy method for modeling subsurface light transport on object surfaces to enable relighting at arbitrary high resolutions. By capturing phase-shifted fringe patterns using a structured light–camera system and integrating 3D geometry as input to a U-Net convolutional neural network, the approach learns per-point, pixel-level footprint responses. It represents the first integration of pixel-level light transport modeling with deep learning, achieving cross-object and multi-view generalization of subsurface scattering appearance. Experimental results demonstrate that the generated relit images exhibit strong qualitative and quantitative agreement with ground-truth photographs and generalize effectively to unseen subsurface scattering materials.

Abstract

We present a method to acquire and estimate the sub-surface scattering properties of light transport at a highly detailed level by learning the pixel footprint response at each point on the object surface. The reconstruction leverages 3D scanning techniques as input to a U-Net CNN. A stereo projector-camera setup using phase-shifted profilometry (PSP) patterns efficiently captures the data for a variety of scattering objects. Reconstructing dense pixel footprints allows for relighting with arbitrary high-resolution projector patterns. The final output is a relit color image. Qualitative and quantitative comparison against illuminated real-world captured images demonstrate that the predicted footprints are almost identical to the actual responses. The same model is trained for multiple views across multiple objects such that the learned representations can be used to generalize to unseen sub-surface scattering materials as well.

Problem

• subsurface scattering
• light transport
• relighting
• pixel footprint
• material generalization

Innovation

• subsurface scattering
• pixel footprint response
• U-Net CNN
• phase-shifted profilometry
• relighting