pose estimation

To address the ill-posedness of scene-level object manipulation reconstruction from monocular RGB video, hand-object depth ambiguity, and lack of physical plausibility, this paper proposes the first zero-shot, scene-centric joint reconstruction framework. Methodologically, it departs from conventional hand-centric paradigms and instead integrates CLIP/SAM/3D diffusion priors for initialization, coupled with differentiable rendering, multi-view geometric constraints, contact-force regularization, and a two-stage co-optimization scheme—enabling simultaneous estimation of hand pose, object deformation and pose, and scene geometry without ground-truth annotations. The approach significantly improves metric consistency and physical realism, achieving centimeter-level accuracy and high temporal coherence even under severe occlusion and dynamic motion. It establishes a novel paradigm for real-scale, joint hand–object–scene inference.

This paper addresses model-free intrinsic camera calibration from a single unstructured outdoor image, without requiring gravity priors, camera-specific assumptions, or external geometric cues. We propose a pixel-ray regression-based intermediate representation that decouples intrinsic estimation into manifold-learning-driven ray modeling and model-agnostic closed-form inversion. The framework uniformly supports diverse projection models—including pinhole, Brown–Conrady, and Kannala–Brandt—and natively accommodates common image edits such as cropping and stretching. Our approach employs only a lightweight neural network, trained on orders-of-magnitude less data than 3D foundation models, yet consistently surpasses state-of-the-art methods in accuracy and robustness. Notably, it demonstrates superior cross-model generalization and resilience to edited images—key challenges in practical deployment.

This work addresses the failure of traditional Structure-from-Motion (SfM) methods in planar scenes due to degeneracy in epipolar geometry. The authors propose a unified framework that leverages homography graph embedding to actively exploit planar structures as geometric constraints. By decomposing multi-view homographies induced by dominant planes, relative camera poses are estimated and organized into a pose graph. Global optimization is then achieved through spectral embedding combined with a maximum spanning tree for consensus maximization. The method significantly outperforms existing SfM approaches in strongly planar environments—such as basketball courts—and achieves state-of-the-art or comparable performance on the IMC Phototourism outdoor benchmark, demonstrating robust camera pose recovery in both degenerate and non-degenerate scenarios.

This work addresses the problem of open-domain single-image 3D geometric reconstruction. To resolve global scale and translation ambiguities, we propose an affine-invariant 3D point cloud representation. Methodologically, we design an optimal point cloud alignment solver and a multi-scale local geometric consistency loss to mitigate the inherent ambiguity of monocular geometric supervision. Our approach integrates affine-invariant representation learning, robust point cloud registration, and end-to-end training on a hybrid large-scale dataset. Extensive experiments demonstrate that our method achieves state-of-the-art performance across multiple unseen benchmarks. It significantly improves accuracy and generalization in monocular 3D point cloud reconstruction, depth estimation, and field-of-view prediction. By eliminating the need for camera calibration or explicit metric priors, our framework establishes a new paradigm for uncalibrated single-image geometric understanding.

Traditional novel view synthesis relies on external Structure-from-Motion (SfM) tools (e.g., COLMAP), which introduce pose estimation errors and computational bottlenecks. To address this, we propose the first end-to-end framework that jointly optimizes 3D Gaussian splatting radiance fields and camera poses. Our method decouples optimization into two alternating stages—Gaussian parameter refinement and pose fine-tuning—guided by differentiable rendering and a geometric-photometric consistency constraint enforced via 3D optical flow. This design significantly suppresses projection errors, especially under large parallax and sparse-texture conditions. Evaluated on multiple standard benchmarks, our approach not only substantially outperforms existing COLMAP-free methods but also surpasses the COLMAP+3DGS baseline across both reconstruction quality and camera pose accuracy.

This work addresses the insufficient integration of learning-based methods and geometric constraints in camera pose and scene structure estimation by proposing a modular framework. The approach first employs a learning model (VGGT) to generate initial hypotheses for depth and relative pose, which are subsequently refined and validated using classical geometric algorithms such as point-to-plane RGB-D ICP. Crucially, the framework explicitly distinguishes the roles of learning as a “proposer” and geometry as a “referee,” emphasizing that the geometric module serves not merely as post-processing but as an essential mechanism for verifying and integrating learned outputs. Experiments on the TUM RGB-D dataset demonstrate that, in moderately challenging rigid scenes, the system significantly outperforms both purely learning-based and purely geometric baselines when the learned depth aligns geometrically with the camera intrinsics and undergoes optimization by the geometric backend.

While existing multi-frame models achieve cross-frame consistency, their single-frame accuracy often lags behind that of single-frame methods. Through systematic ablation studies, this work demonstrates that data diversity and quality are critical for 3D geometry estimation and reveals that commonly used loss functions may inadvertently suppress performance. To address these issues, the authors propose CARVE, a novel approach integrating a high-resolution network architecture, joint sequence- and frame-level supervision, a consistency loss, and alignment between depth maps and camera parameters. CARVE achieves state-of-the-art and robust performance across multiple benchmarks in tasks including point cloud reconstruction, video depth estimation, and estimation of camera pose and intrinsics.

This work addresses the challenge of certifiable global optimality in non-convex geometric optimization for 3D vision by systematically reviewing and unifying three global solver paradigms—Branch-and-Bound (BnB), Convex Relaxation (CR), and Graduated Non-Convexity (GNC)—across ten core tasks, including the Wahba problem and bundle adjustment. It establishes the first comprehensive taxonomy and unified framework for global optimization in 3D vision, elucidating the fundamental trade-offs among optimality, robustness, and scalability. The study further outlines a promising direction that integrates data-driven priors with certifiable optimization. By providing a trustworthy perception roadmap for safety-critical applications, this work also contributes an open-source, continuously updated survey and code tutorial to foster reproducibility and community advancement.

Existing stereo vision models suffer from severe degradation of geometric information during feature extraction due to the absence of explicit camera pose supervision. This work proposes StereoVGGT, which, for the first time, adapts a frozen pre-trained Visual Geometry Group Transformer (VGGT) to stereo matching without any additional training. By incorporating binocular geometric constraints, StereoVGGT effectively reconstructs feature representations to preserve and leverage the model’s intrinsic camera calibration and 3D geometric priors. The method achieves state-of-the-art performance on the KITTI benchmark, surpassing all previously published approaches and securing the top rank.

Existing pose-agnostic feedforward 3D Gaussian splatting methods couple geometry and appearance modeling, which constrains reconstruction quality. This work proposes 2Xplat, a novel framework that introduces, for the first time within this paradigm, a decoupled dual-expert architecture: a geometry expert explicitly predicts camera poses, while an appearance expert leverages these poses to generate high-fidelity 3D Gaussian representations. This modular design challenges the prevailing end-to-end integrated paradigm and significantly outperforms existing pose-agnostic approaches in fewer than 5,000 training iterations. Remarkably, 2Xplat achieves performance on par with state-of-the-art methods that rely on known camera poses, thereby demonstrating the efficacy and advantages of decoupled modeling for complex 3D reconstruction tasks.