A Multi-Layer System for Ultra-High-Resolution Static 360-Degree Telepresence

2026-08-05

AI Summary

This work addresses the limitations of existing 360-degree telepresence systems, which struggle to simultaneously preserve high-fidelity panoramic detail and support interactivity in static scenes due to hardware resolution constraints. To overcome this, the authors propose a multi-layer representation framework tailored for static environments: a base layer comprising an 8K omnidirectional background captured by a panoramic camera and offline stitched for high fidelity; a middle layer enabling real-time foreground synthesis via a 4K pan-tilt-zoom (PTZ) camera; and a top layer facilitating user-guided streaming of 4K region-of-interest video. By integrating tiled background modeling, real-time high-resolution matting, and an online updating mechanism, the system significantly enhances visual detail and immersion. User studies demonstrate that the proposed approach outperforms current video super-resolution methods in perceived quality, effectively achieving a balance between high fidelity and interactivity in static 360-degree telepresence.

Abstract

360-degree video telepresence offers strong immersive potential but remains constrained by the limited resolution of current capture and display hardware. Many telepresence installations feature fixed viewpoints and largely static scenes, yet optimization strategies tailored to such setups have received limited attention. We present a multi-layer, ultra-high-resolution system for static 360-degree telepresence that combines an 8K panoramic camera with a rotatable 4K pan-tilt-zoom (PTZ) camera. Our approach builds a three-layer representation: (1) a tile-based ultra-high-resolution panoramic background, generated by offline stitching high-detail 4K PTZ scans onto the base 8K panorama to achieve effective resolution beyond native capture, and represented as a set of spatial tiles; (2) a dynamic update layer that composites foreground motions from the 8K stream via real-time high-resolution background matting; and (3) a region-of-interest 4K layer that streams a real-time PTZ view of the selected region and additionally updates the corresponding background tiles over time. We evaluate the proposed system through comparisons with representative video super-resolution approaches and a user study assessing perceived detail and immersive experience. Our results indicate that tile-based background refinement, together with user-guided updates, provides a practical way to balance panoramic fidelity and interactivity in static 360-degree telepresence.

Problem

• 360-degree telepresence
• ultra-high-resolution
• static scenes
• panoramic video
• telepresence systems

Innovation

• multi-layer representation
• ultra-high-resolution 360-degree telepresence
• tile-based background refinement
• PTZ camera integration
• background matting