TensoIS: A Step Towards Feed-Forward Tensorial Inverse Subsurface Scattering for Perlin Distributed Heterogeneous Media

2025-09-04

AI Summary

This paper addresses the severely ill-posed inverse problem of recovering 3D subsurface scattering parameters of heterogeneous media from sparse multi-view images. To tackle this challenge, we propose the first learning-based feedforward framework. Our method introduces three key innovations: (1) Fractal Perlin noise is employed—novelly—to model realistic heterogeneous scattering distributions, enabling synthesis of the HeteroSynth dataset; (2) a learnable low-rank tensor representation is designed to encode the scattering field compactly, efficiently, and differentiably; and (3) differentiable rendering is integrated with supervised learning to establish an end-to-end mapping from input images to 3D scattering parameters. Extensive evaluation demonstrates strong generalization across unseen geometries, synthetic smoke/cloud simulations, and real-world samples, achieving significant improvements in estimation accuracy and robustness. This work establishes a new paradigm for inverse scattering in complex heterogeneous media.

Technology Category

• Search and Optimization: Learning to Search
• Machine Learning: Learning with Manifolds
• Computer Vision: Computational Photography, Image & Video Synthesis

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Estimating scattering parameters of heterogeneous media from images is a severely under-constrained and challenging problem. Most of the existing approaches model BSSRDF either through an analysis-by-synthesis approach, approximating complex path integrals, or using differentiable volume rendering techniques to account for heterogeneity. However, only a few studies have applied learning-based methods to estimate subsurface scattering parameters, but they assume homogeneous media. Interestingly, no specific distribution is known to us that can explicitly model the heterogeneous scattering parameters in the real world. Notably, procedural noise models such as Perlin and Fractal Perlin noise have been effective in representing intricate heterogeneities of natural, organic, and inorganic surfaces. Leveraging this, we first create HeteroSynth, a synthetic dataset comprising photorealistic images of heterogeneous media whose scattering parameters are modeled using Fractal Perlin noise. Furthermore, we propose Tensorial Inverse Scattering (TensoIS), a learning-based feed-forward framework to estimate these Perlin-distributed heterogeneous scattering parameters from sparse multi-view image observations. Instead of directly predicting the 3D scattering parameter volume, TensoIS uses learnable low-rank tensor components to represent the scattering volume. We evaluate TensoIS on unseen heterogeneous variations over shapes from the HeteroSynth test set, smoke and cloud geometries obtained from open-source realistic volumetric simulations, and some real-world samples to establish its effectiveness for inverse scattering. Overall, this study is an attempt to explore Perlin noise distribution, given the lack of any such well-defined distribution in literature, to potentially model real-world heterogeneous scattering in a feed-forward manner.

Problem

• Estimating heterogeneous subsurface scattering parameters from images
• Modeling real-world scattering with Perlin noise distribution
• Developing feed-forward learning framework for inverse scattering

Innovation

• Uses Fractal Perlin noise to model heterogeneous scattering parameters
• Proposes feed-forward learning framework with low-rank tensor representation
• Creates synthetic dataset HeteroSynth for training and evaluation