CAM3R: Camera-Agnostic Model for 3D Reconstruction

2026-03-23

AI Summary

This work addresses the significant performance degradation of existing 3D reconstruction methods on non-pinhole imagery—such as fisheye or panoramic views—due to their reliance on the pinhole camera assumption. To overcome this limitation, we propose the first feed-forward, calibration-free, universal 3D reconstruction framework capable of handling diverse camera models. Our approach employs a dual-branch network to jointly estimate per-pixel ray directions and radial distances, complemented by a ray-aware global alignment mechanism that fuses local geometric cues while simultaneously optimizing pose and scale. Extensive experiments demonstrate that our method achieves state-of-the-art performance in both 3D reconstruction and pose estimation across fisheye, panoramic, and pinhole image datasets, marking the first unified, calibration-free solution for cross-camera-model 3D reconstruction.

Technology Category

• Computer Vision: 3D Computer Vision
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Machine Learning: Calibration & Uncertainty Quantification

Application Category

• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation

Abstract

Recovering dense 3D geometry from unposed images remains a foundational challenge in computer vision. Current state-of-the-art models are predominantly trained on perspective datasets, which implicitly constrains them to a standard pinhole camera geometry. As a result, these models suffer from significant geometric degradation when applied to wide-angle imagery captured via non-rectilinear optics, such as fisheye or panoramic sensors. To address this, we present CAM3R, a Camera-Agnostic, feed-forward Model for 3D Reconstruction capable of processing images from wide-angle camera models without prior calibration. Our framework consists of a two-view network which is bifurcated into a Ray Module (RM) to estimate per-pixel ray directions and a Cross-view Module (CVM) to infer radial distance with confidence maps, pointmaps, and relative poses. To unify these pairwise predictions into a consistent 3D scene, we introduce a Ray-Aware Global Alignment framework for pose refinement and scale optimization while strictly preserving the predicted local geometry. Extensive experiments on various camera model datasets, including panorama, fisheye and pinhole imagery, demonstrate that CAM3R establishes a new state-of-the-art in pose estimation and reconstruction.

Problem

• 3D reconstruction
• wide-angle imagery
• camera-agnostic
• geometric degradation
• unposed images

Innovation

• Camera-Agnostic
• 3D Reconstruction
• Wide-Angle Imaging
• Ray-Based Geometry
• Global Alignment