🤖 AI Summary
This work addresses the fundamental challenge in panoramic scene understanding: geometric distortions induced by spherical-to-planar projection violate the translation invariance assumption of conventional vision models, making it difficult to simultaneously achieve strict spherical equivariance and reuse pre-trained model weights. The study systematically establishes a two-dimensional taxonomy encompassing architectural design and training paradigms, revealing this core tension for the first time. It identifies five systemic deficiencies in current evaluation protocols and proposes a six-point research roadmap toward general-purpose panoramic intelligence. By integrating spherical geometric modeling, distortion-aware networks, native spherical operators, and geometry-aware tokenization, the authors construct a unified framework that pinpoints critical research gaps, advocates for standardized benchmarking, and releases an open-source survey repository to foster community progress.
📝 Abstract
Panoramic images capture the complete visual sphere in a single frame, providing spatial context unattainable by conventional cameras. Yet this completeness comes at a geometric cost: the 2-sphere cannot be faithfully mapped to the plane, and every planar representation introduces distortions that violate the assumptions underlying standard vision architectures. This survey traces the evolution of panoramic scene analysis along a methodological trajectory, from projection-based adaptation, through distortion-aware engineering, to sphere-native modeling and geometry-aware tokenization for foundation models, and argues that this evolution reflects a progressive deepening of geometric commitment rather than a simple accumulation of techniques. We organize the literature along two orthogonal dimensions: architectural design (how operators interact with spherical geometry) and training paradigm (how knowledge is transferred across domains). Covering dense prediction (semantic segmentation, depth estimation, and room layout estimation), unified multi-task understanding, open-world perception, vision-language reasoning, and dynamic video analysis, we identify a central unresolved tension: among the methods surveyed, none simultaneously delivers strict spherical equivariance and full reuse of perspective-pretrained foundation-model weights, and we argue that this is a structural rather than incidental gap. We further expose five systematic gaps in current evaluation protocols, namely the absence of spherical-area-weighted metrics, seam-consistency testing, polar-robustness stratification, cross-projection generalization, and open-world protocol standardization, and propose a six-point research roadmap toward general-purpose panoramic intelligence. The corresponding repository is publicly available at: https://github.com/zhuqinfeng1999/Awesome-Panoramic-Scene-Analysis.