thermomechanical gaussian splatting

Existing neural rendering approaches generally overlook the influence of temperature on visual appearance, making it difficult to faithfully simulate thermally coupled phenomena such as melting and solidification. This work introduces thermal phase-change dynamics into 3D Gaussian splatting for the first time by endowing Gaussian points with temperature attributes. It couples a numerical heat conduction–convection solver with Material Point Method (MPM) dynamics and proposes a topology-adaptive rendering strategy to mitigate visual artifacts caused by large deformations. The resulting framework achieves high-fidelity, physically consistent rendering of thermophysical dynamic scenes, demonstrating significantly improved realism and controllability over existing methods in complex phase-transition processes like melting and solidification.

该研究提出了一种动态RGB-热成像重建框架，通过多模态动态场景表示和自适应生成特定模态高斯分布的方法解决了温度分布随时间变化的问题。

This work addresses key challenges in thermal-infrared (T) and RGB dual-modal 3D scene reconstruction: cross-modal misalignment, single-modal overfitting, and low-fidelity real-time rendering. We propose the first 3D Gaussian Splatting framework tailored for thermal imaging. Methodologically, we introduce a thermal-aware 3D Gaussian representation; incorporate multi-modal regularization and thermal-physical priors into a smoothness constraint; and construct the first handheld real-world RGBT-Scenes dataset. Our contributions are threefold: (1) the first extension of 3D Gaussian Splatting to thermal imaging, enabling photorealistic thermal map rendering while simultaneously improving RGB reconstruction quality; (2) a 90% reduction in model size, enabling real-time dual-modal rendering; and (3) significant improvements in cross-modal geometric-radiometric consistency and generalization over existing single-modal approaches.

This work addresses the limitation of existing physics-based 3D simulation methods—where Gaussian splatting (GS) is coupled with dynamics only via explicit surface meshes (e.g., Marching Cubes)—by proposing the first mesh-free, differentiable physical modeling framework for GS. Methodologically, it treats 3D Gaussian primitives as discrete Newtonian point masses governed by physical dynamics; Gaussian distributions are flattened and deformed via triangle-mesh parameterization, enabling plug-and-play integration with arbitrary black-box physics engines. Key contributions include: (1) the first end-to-end differentiable, mesh-free Gaussian physical model; (2) a novel patch-driven deformation mechanism that jointly preserves geometric fidelity and dynamic consistency; and (3) significant improvements in visual quality and physical plausibility across multiple 3D object rendering benchmarks, while maintaining real-time rendering efficiency.

This work presents a systematic survey of 3D Gaussian Splatting (3D GS), clarifying its theoretical foundations as an explicit radiance field representation paradigm, its key technical advancements, and its practical applicability boundaries. We introduce the first 3D GS knowledge graph, elucidating its intrinsic mechanisms—explicit spatial parameterization, differentiable rasterization, and strong editability—and rigorously distinguishing it from implicit NeRF-based approaches. Methodologically, we integrate adaptive Gaussian optimization, density-aware regularization, and multi-view geometric constraints to enable end-to-end training and real-time rendering (>100 FPS). Comprehensive experiments benchmark state-of-the-art models across reconstruction accuracy, inference speed, and editing flexibility, revealing critical limitations—including low geometric fidelity and weak support for dynamic scenes. Finally, we identify promising future directions: scalability to large-scale scenes, physical plausibility enforcement, and cross-modal integration with vision-language or sensor-fusion frameworks.

Existing 3D Gaussian splatting methods struggle to simulate brittle fracture due to the absence of a structurally coherent volumetric interior representation and fracture-aware physical simulation mechanisms. This work proposes a unified framework that, for the first time, enables coherent internal modeling with realistic textures and dynamic fracture simulation within a Gaussian representation. The approach leverages a generative model to synthesize plausible multi-material internal structures and integrates an optimized Continuum Damage Material Point Method (CD-MPM) to drive fracture propagation, all embedded within a 3D Gaussian splatting rendering pipeline. The method supports multi-stage fracture in complex, heterogeneous material scenes, achieving photorealistic visual quality and real-time performance. It significantly outperforms existing techniques and is well-suited for interactive applications such as virtual reality and robotics.

This work addresses the key challenge of recovering spatially varying thermophysical properties of complex 3D scenes from thermal imaging observations, a capability critical for applications such as digital twins and infrastructure monitoring. To this end, the authors propose ThermoField, a novel framework that, for the first time, integrates neural scene representations with a differentiable heat conduction solver. By modeling both geometry and spatially varying thermal diffusivity through neural fields and leveraging time-resolved thermal observations, the method enables physics-guided joint optimization. ThermoField unifies high-fidelity geometric reconstruction, accurate estimation of thermophysical parameters, and predictive simulation of thermal evolution under novel environmental conditions. Extensive experiments on both real-world and synthetic datasets demonstrate its effectiveness and strong generalization across diverse environments.

This work systematically investigates the semantic representation capacity of 3D Gaussian splatting for scene understanding. For this emerging representation, it presents the first comprehensive evaluation of geometric deep learning architectures—including point cloud networks and graph neural networks—on scene classification tasks, employing a combination of end-to-end training, linear probing, and clustering analysis. The study reveals significant performance disparities across model families when applied to Gaussian splatting data and demonstrates that geometry- and appearance-specific attributes inherent to Gaussians—such as covariance and opacity—substantially enhance representation quality. These findings establish a foundational basis for future semantic understanding tasks leveraging Gaussian splatting representations.

Existing NeRF and 3D Gaussian Splatting (3DGS) methods achieve high fidelity in RGB reconstruction but lack physically grounded modeling for thermal infrared modalities—ignoring critical thermal phenomena such as heat conduction, Lambertian radiation, and radiative decay. This work introduces the first RGB-thermal joint reconstruction framework built upon 3DGS. It unifies visible and thermal imaging through orthogonal feature disentanglement and view-dependent embedding, while incorporating physics-based priors: Fourier-based heat conduction, the Stefan–Boltzmann law, and inverse-square radiative attenuation—to generate depth-aware, physically consistent thermal radiance maps. The method achieves high-fidelity, geometrically coherent, and physically plausible cross-modal novel-view synthesis, significantly reducing Gaussian count without sacrificing quality. Experimental results demonstrate breakthrough improvements in both photorealism and physical consistency, establishing a new benchmark for multimodal neural scene representation with explicit thermal physics.

This work addresses the challenge of efficiently constructing 3D radiance fields in thermal infrared scenes without relying on visible-light or multimodal data. It proposes Thermal-to-Depth Gaussian Splatting (TDg), a novel method that, for the first time, enables high-quality 3D reconstruction using only monomodal thermal inputs. TDg leverages monocular depth estimation to guide geometric modeling within the 3D Gaussian splatting framework, achieving competitive rendering fidelity while substantially reducing computational overhead. Experimental results on the RGBT-Scenes and ThermalMix datasets demonstrate that TDg outperforms the MSMG baseline across key perceptual and photometric metrics—LPIPS, SSIM, and PSNR—and reduces training time by 55% (equivalent to 12 minutes and 47 seconds), thereby validating its efficiency and practicality for thermal-only 3D scene reconstruction.