LVSPM: Long Sequence View Synthesis and Pose Estimation Model

2026-10-07

AI Summary

This work addresses the challenges of camera pose estimation and novel view synthesis from uncalibrated image sets by proposing a general-purpose model, LVSPM. Adopting an end-to-end architecture, the method jointly optimizes both tasks using only RGB images and pose supervision, eliminating the need for dense 3D ground truth. Furthermore, it incorporates a test-time training mechanism and a multi-dataset joint strategy to overcome long-sequence scalability bottlenecks, enabling seamless processing of hundreds of input views. Experiments demonstrate that LVSPM achieves leading pose estimation performance on benchmarks such as RealEstate10k and delivers state-of-the-art novel view synthesis quality, significantly outperforming VGGT and pose-dependent baselines while maintaining stability as scene scale increases. The source code has been made publicly available.

Abstract

We present LVSPM, a generalizable model that jointly estimates camera poses and synthesizes novel views from uncalibrated image collections. Trained with only RGB images and pose supervision, LVSPM avoids dense 3D ground truth and employs test-time training (TTT) layers to scale seamlessly to hundreds of input views. On RealEstate10k, Co3Dv2, and DL3DV, LVSPM surpasses VGGT in pose estimation across 16-256 views, with especially large margins at strict thresholds. For novel view synthesis under a practical protocol where more views cover larger scenes, LVSPM achieves state-of-the-art pose-free quality---surpassing even pose-dependent models in PSNR---and still maintains high quality as scene scale grows, while baselines collapse. The code is available at https://burningdust21.github.io/Projects/LVSPM .

Problem

• novel view synthesis
• camera pose estimation
• uncalibrated images
• long sequence
• pose-free

Innovation

• Long Sequence View Synthesis
• Pose Estimation
• Test-Time Training
• Novel View Synthesis
• Generalizable Model