HybridSim: A Physics-Learning Hybrid Digital Twin for mmWave Human Sensing

2026-07-17

AI Summary

This work addresses the challenge of high-fidelity millimeter-wave radar signal simulation for dynamic human sensing, where acquiring annotated real-world data is prohibitively expensive. To this end, the authors propose a hybrid digital twin framework that integrates physical modeling with learning-based techniques to synthesize radar signals from dynamic human meshes. The approach decouples propagation paths into direct and indirect components, modeling them separately for greater fidelity. It innovatively combines tri-plane feature representations with graph convolutional networks to stabilize optimization and integrates microfacet BRDF inverse rendering with 3D Gaussian splatting to significantly reduce the computational cost of multipath simulation while preserving accuracy. Experimental results demonstrate that the synthesized radar signals exhibit strong consistency with physical measurements and consistently enhance performance in downstream human perception tasks through effective data augmentation.

Abstract

High-fidelity simulation of mmWave radar signals for dynamic human motion is valuable for developing radar-based human sensing models; yet collecting accurately labeled measurements for a specific deployment site remains expensive. We present HybridSim, a physics-learning hybrid simulator that synthesizes mmWave radar signals from dynamic human meshes under a fixed indoor room configuration, explicitly decoupling propagation into two components. To parameterize the human subject, we use a tri-plane representation to extract human features and a Graph Convolutional Network to stabilize optimization and mitigate gradient instability. The direct signal path is modeled via an inverse-rendering formulation with a microfacet BRDF to capture primary surface reflections. In parallel, the indirect path is approximated by combining 3D Gaussian Splatting with a virtual-receiver geometry to fit and reproduce site-specific multipath interference patterns, achieving substantially lower computational cost than explicit full ray tracing. Experiments in a fixed-room setting show improved agreement with a physically based reference and consistent gains on downstream radar-based human sensing tasks when using HybridSim for site-specific data augmentation.

Problem

• mmWave radar
• human sensing
• digital twin
• signal simulation
• data augmentation

Innovation

• Hybrid Digital Twin
• mmWave Radar Simulation
• 3D Gaussian Splatting
• Inverse Rendering
• Multipath Modeling