monocular depth estimation

Existing novel view synthesis methods heavily rely on camera calibration or geometric priors, limiting scalability to large-scale uncalibrated video data. To address this, we propose the first fully uncalibrated, purely self-supervised two-stage framework. In the first stage, scene and camera representations are jointly optimized in an implicit latent space under multi-view consistency constraints. In the second stage, 3D Gaussian primitives are introduced, integrated with differentiable Gaussian splatting rendering and a depth-projection loss to explicitly align the implicit representation with real-world 3D geometry. Our method trains end-to-end solely from raw video frames or multi-view images—without any camera intrinsics, extrinsics, or depth supervision. It achieves high-fidelity novel view synthesis and accurate camera pose estimation, matching state-of-the-art performance of strongly supervised approaches.

To address the ill-posedness of scene-level object manipulation reconstruction from monocular RGB video, hand-object depth ambiguity, and lack of physical plausibility, this paper proposes the first zero-shot, scene-centric joint reconstruction framework. Methodologically, it departs from conventional hand-centric paradigms and instead integrates CLIP/SAM/3D diffusion priors for initialization, coupled with differentiable rendering, multi-view geometric constraints, contact-force regularization, and a two-stage co-optimization scheme—enabling simultaneous estimation of hand pose, object deformation and pose, and scene geometry without ground-truth annotations. The approach significantly improves metric consistency and physical realism, achieving centimeter-level accuracy and high temporal coherence even under severe occlusion and dynamic motion. It establishes a novel paradigm for real-scale, joint hand–object–scene inference.

Reconstructing high-quality, freely renderable dynamic 3D scenes from monocular videos remains challenging due to occlusions, missing viewpoints, and motion blur. This work proposes a novel approach that leverages generative video models to refine and complete an initial dynamic 3D Gaussian Splatting (3DGS) reconstruction. By guiding the generative model with densely pixel-aligned renderings and distilling the synthesized content back into a unified dynamic 3DGS representation, the method achieves consistent and high-fidelity 4D reconstructions from single-view videos. It supports large viewpoint variations and complex motions, establishing a new state of the art in 4D reconstruction. The approach significantly improves novel view synthesis quality and 3D motion accuracy, and demonstrates strong generalization to real-world in-the-wild videos.

To address geometric reasoning from unlabeled RGB-D data, this paper proposes a self-supervised point cloud registration framework. The method introduces (1) a cycle-consistent keypoint selection mechanism that enforces cross-view spatial constraints via geometrically salient anchor points, and (2) a pose estimation module integrating GRU-based temporal modeling with transformation synchronization, jointly leveraging historical observations and multi-view geometric consistency. Without requiring any manual annotations, the framework achieves state-of-the-art performance among self-supervised approaches on ScanNet and 3DMatch—matching or even surpassing certain fully supervised baselines. It further demonstrates strong generalizability: the learned representations can be readily integrated into downstream SLAM or reconstruction systems, significantly improving their robustness and accuracy.

This work addresses the challenge of occlusion inpainting in monocular-to-stereoscopic video conversion, a task hindered by the limited availability of real stereo data and domain shifts inherent in synthetic datasets, which constrain model generalization. The paper introduces the first self-supervised framework based on cycle consistency that learns to generate high-quality stereoscopic videos from ordinary monocular videos without requiring ground-truth stereo pairs. Its key innovation lies in incorporating the Geometric Reciprocity Theorem (GRT), which enables, for the first time, the analytical computation of occlusion masks at test time directly from monocular inputs, thereby ensuring consistency between training and inference under a nearest-neighbor depth-image-based rendering (DIBR) setup. Experiments demonstrate that the proposed method significantly outperforms both unsupervised and supervised baselines under fully unsupervised conditions, offering an effective pathway to leverage the vast abundance of monocular video content for stereoscopic generation.

This work addresses the geometric inconsistency and drift commonly observed in existing self-supervised monocular video depth estimation methods, which stem from a lack of global 3D awareness. The authors reformulate the task as an unconstrained multi-view 3D reconstruction problem and introduce, for the first time, geometric priors from a 3D foundation model to establish a unified framework enforcing 3D consistency. This framework jointly optimizes three complementary constraints: image-level photometric rendering, explicit world-coordinate geometric alignment, and multi-scale temporal gradient consistency, thereby enhancing both inter-frame coherence and spatial accuracy. Experiments demonstrate that the proposed method significantly outperforms current depth estimation and 3D reconstruction baselines under both self-supervised training and zero-shot clinical scenarios.

This work addresses the challenge of learning interpretable and structured 3D scene representations from RGB-D or voxel data under fully unsupervised conditions, with the goal of supporting downstream tasks such as robotic manipulation. To this end, the authors propose a self-supervised method that decomposes a scene into a set of implicit particles, each endowed with disentangled attributes—namely 3D keypoints, size, and appearance—and generates corresponding segmentation masks through end-to-end reconstruction. This approach is the first to extend object-centric representations to 3D while enabling fully self-supervised training. The learned particles exhibit both interpretability and controllability, facilitating scene editing and efficient inference. Experiments demonstrate that the proposed representation significantly outperforms baseline methods that either rely on dense 3D inputs or lack structural priors, leading to improved performance in robotic manipulation tasks on both simulated and real-world data.

This work proposes NAS3R, a novel framework that achieves end-to-end self-supervised 3D Gaussian reconstruction from uncalibrated, pose-free 2D images without requiring ground-truth 3D annotations or pretrained priors. NAS3R jointly learns explicit 3D geometry and camera parameters through a shared Transformer backbone, incorporating a mask attention mechanism and depth-guided 3D Gaussian representations, optimized solely with 2D photometric supervision. Experiments demonstrate that NAS3R outperforms existing self-supervised methods across multiple benchmarks, attaining geometric accuracy comparable to supervised approaches, while remaining flexible enough to integrate known intrinsics or prior information when available.

Existing methods struggle to jointly model 3D geometry and view-dependent appearance effects—such as specular highlights and Fresnel reflections—within a unified framework. This work proposes a novel implicit 3D representation that, for the first time, encodes stochastically subsampled surface light fields into compact latent vectors, enabling a unified latent space that co-represents both geometry and view-dependent appearance. Leveraging a single input image, the method employs an implicit flow-matching model to generate 3D objects with consistent illumination and material properties. The approach outperforms prior work in both visual realism and input fidelity, accurately reproducing complex view-dependent effects under challenging lighting conditions.

This work investigates whether self-supervised vision foundation models learn feature representations that align with the intrinsic structure of three-dimensional Euclidean space, even without explicit 3D supervision. To this end, we introduce a novel probing methodology based on neighborhood alignment and Poincaré adapters, along with a “latent space navigation” technique that leverages the topological and geometric relationship between the feature space and the SE(3) group to enable visual odometry and localization without explicit 3D reconstruction. Our experiments demonstrate a strong correspondence between the model’s latent subspace and 3D spatial structure, achieving accurate motion estimation and localization in static scenes using only latent features—an outcome that provides the first empirical validation of implicit 3D structural awareness in self-supervised visual representations.