SPW-Nav: A Streaming Panoramic World Model for Language-Guided Navigation

2026-10-06

AI Summary

This study addresses the limitations of predefined trajectories in panoramic generation and the inability of low-view models to respond to language instructions in real time. To this end, it proposes a streaming panoramic world model that parses language instructions into camera motions, generating one-minute 2K 360-degree videos from a single panoramic image in real time. Methodologically, spherical rotation decoupling, pose-aligned conditioning, and a multi-term memory mechanism are designed to support dynamic instruction switching while preserving long-sequence consistency. Efficiency is further enhanced through spherical geometry processing and a few-step generation architecture, alongside the construction of the SPW-NavSet dataset containing verified instructions. Experimental results demonstrate that the proposed model surpasses existing methods in both camera-following accuracy and video quality, achieving real-time interactive panoramic video generation.

Abstract

Language-guided panoramic video generation benefits various downstream applications, such as interactive 3D scene exploration, virtual reality experiences, and embodied agent training. Existing panoramic generators follow predefined trajectories, and interactive world models act through low-level actions in perspective views. We propose SPW-Nav, a streaming panoramic world model that understands movement instructions and streams one minute of 2K 360-degree video in real time from a single panorama. SPW-Nav interprets each instruction in the previously generated panorama as camera motion. Spherical rotation decoupling applies rotation exactly on the sphere, pose-aligned conditioning keeps translation inputs bounded over long streams, and a multi-term memory with a few-step generator continues the scene as instructions change. We also build SPW-NavSet, panoramic videos with camera trajectories and verified instructions. Driven by language, SPW-Nav outperforms prior panoramic generators in camera-following accuracy and video quality, and supports on-the-fly instruction switching.

Problem

• panoramic video generation
• language-guided navigation
• world model
• streaming generation

Innovation

• Streaming Panoramic World Model
• Language-Guided Navigation
• Spherical Rotation Decoupling
• Pose-Aligned Conditioning
• 360-Degree Video Generation