TAP3D: Thermal-Assisted 3D Human Point Clouds

2026-10-08

AI Summary

This study addresses the high cost, sparse deployment, and privacy concerns of conventional sensors by proposing a novel paradigm for 3D human point cloud reconstruction using body heat. Methodologically, it integrates a forward thermal physics model with multi-primitive estimation and geometric perspective fusion algorithms, employing self-supervised learning to recover depth and disentangle multi-person interference from low-resolution commercial thermal array signals. The authors construct a large-scale thermal signal dataset to enable high-fidelity dense point cloud generation. The proposed approach demonstrates strong performance in fall detection (91.46% accuracy) and indoor tracking (21.86 cm error), effectively balancing privacy preservation with dense spatial perception. The source code has been made publicly available.

Abstract

Human body point clouds are a versatile representation for AI-enabled human sensing. However, existing methods using LiDAR, radar, and depth cameras suffer from inherent drawbacks in high cost, sparse reconstruction, and privacy concerns, etc. In this paper, we exploit low-cost thermal arrays and present TAP3D, the first system to reconstruct 3D human point clouds from body heat signatures, offering significant advantages in cost, density, human sensitivity, and privacy. To overcome major challenges in depth estimation, thermal interference, and multi-person separation, we propose a novel physics-informed design, which integrates a forward thermal physics model with two distinct modules: multi-primitive estimation for self-supervised joint recovery of depth and other thermal properties, and geometric perspective fusion for suppressing interference and disentangling multiple people. We implement TAP3D using a single commodity thermal array sensor and build a large-scale dataset (160K samples, 8 environments, 11 users) for evaluation. TAP3D achieves remarkable accuracy for dense point cloud generation, enabling downstream tasks like fall detection (91.46%), indoor tracking (21.86 cm MAE), and human mesh recovery (4.87 cm error). By transforming body heat into point clouds for the first time, TAP3D pioneers a new paradigm for privacy-first, fully passive human sensing for many applications. TAP3D is open-sourced at https://github.com/aiot-lab/TAP3D.

Problem

• 3D human point clouds
• thermal sensing
• privacy-preserving
• human sensing
• depth estimation

Innovation

• Thermal-Assisted 3D Reconstruction
• Physics-Informed Design
• Self-Supervised Depth Estimation
• Human Point Clouds
• Privacy-Preserving Sensing