DynaTok: Token-Based 4D Reconstruction from Partial Point Clouds

2026-06-10

AI Summary

This work proposes the first purely point-cloud-based 4D reconstruction framework that operates without images or explicit correspondences, addressing the challenge of reconstructing complete and temporally coherent 4D sequences from incomplete, unordered point cloud observations lacking explicit temporal alignment. The method employs a Transformer-based spatiotemporal encoder that unifies multi-frame observations through latent tokens and introduces a novel residual token mechanism to disentangle geometric structure from dynamic motion. Coupled with a flow-matching decoder, the model generates high-fidelity, temporally consistent 4D point cloud sequences. Extensive experiments on both object- and scene-level benchmarks demonstrate significant improvements over existing approaches, achieving state-of-the-art performance in reconstruction quality and temporal coherence.

Abstract

We address 4D reconstruction from partial point cloud sequences, where depth-sensor observations are incomplete, unordered, and lack explicit temporal correspondences. This geometry-only setting is challenging due to missing observations and ambiguous dynamics. While recent progress has largely relied on image-based methods, existing point-based approaches typically focus on single objects, assume relatively complete inputs, or require explicit correspondences. To address these limitations, we propose DynaTok, a point-based framework for correspondence-free 4D reconstruction from partial point cloud sequences without images. DynaTok encodes frames into compact latent tokens, aggregates incomplete observations over time with a Transformer-based spatiotemporal encoder, and decouples geometry and motion through residual tokens in a unified model. A flow-matching decoder then reconstructs complete, temporally consistent 4D point-cloud sequences conditioned on the latent tokens. Experiments on object- and scene-level benchmarks demonstrate improved reconstruction quality and temporal coherence from partial point cloud observations. Project page: https://wrchen530.github.io/dynatok/.

Problem

• 4D reconstruction
• partial point clouds
• temporal correspondence
• point cloud sequences
• geometry-only setting

Innovation

• 4D reconstruction
• partial point clouds
• correspondence-free
• Transformer-based spatiotemporal modeling
• flow-matching