DP-DeGauss: Dynamic Probabilistic Gaussian Decomposition for Egocentric 4D Scene Reconstruction

2026-04-09

AI Summary

This work addresses the challenge of 4D reconstruction in egocentric dynamic scenes, where complex camera motion, occlusions, and hand–object interactions hinder effective disentanglement of background, hands, and objects. To overcome the limitations of existing methods that treat all dynamic content as a monolithic foreground, we propose a dynamic probabilistic Gaussian decomposition framework. Starting from a unified 3D Gaussian set initialized via COLMAP, each Gaussian is assigned learnable class probabilities and dynamically routed—through a dedicated routing mechanism—to specialized deformation branches for background, hands, or objects. Fine-grained disentanglement is achieved by jointly optimizing category masks, photometric consistency, and optical flow constraints. Our approach is the first to enable explicit, fine-grained separation of these three components. Experiments demonstrate state-of-the-art performance, with consistent improvements in PSNR (+1.70 dB), SSIM, and LPIPS.

Abstract

Egocentric video is crucial for next-generation 4D scene reconstruction, with applications in AR/VR and embodied AI. However, reconstructing dynamic first-person scenes is challenging due to complex ego-motion, occlusions, and hand-object interactions. Existing decomposition methods are ill-suited, assuming fixed viewpoints or merging dynamics into a single foreground. To address these limitations, we introduce DP-DeGauss, a dynamic probabilistic Gaussian decomposition framework for egocentric 4D reconstruction. Our method initializes a unified 3D Gaussian set from COLMAP priors, augments each with a learnable category probability, and dynamically routes them into specialized deformation branches for background, hands, or object modeling. We employ category-specific masks for better disentanglement and introduce brightness and motion-flow control to improve static rendering and dynamic reconstruction. Extensive experiments show that DP-DeGauss outperforms baselines by +1.70dB in PSNR on average with SSIM and LPIPS gains. More importantly, our framework achieves the first and state-of-the-art disentanglement of background, hand, and object components, enabling explicit, fine-grained separation, paving the way for more intuitive ego scene understanding and editing.

Problem

• egocentric 4D reconstruction
• dynamic scene decomposition
• hand-object interaction
• occlusion handling
• ego-motion

Innovation

• Dynamic Probabilistic Gaussian Decomposition
• Egocentric 4D Reconstruction
• Category-aware Routing
• Hand-Object Disentanglement
• Deformation Branches