GazeOnce360: Fisheye-Based 360° Multi-Person Gaze Estimation with Global-Local Feature Fusion

2026-03-17

AI Summary

This work addresses the challenge of 360° multi-person 3D gaze estimation from an upward-facing fisheye camera view by proposing GazeOnce360, an end-to-end model that integrates global low-resolution contextual information with local high-resolution eye details through a dual-resolution architecture. To handle fisheye distortion and viewpoint variations, the model incorporates rotation-equivariant convolutions and leverages eye landmark supervision to enhance estimation accuracy. The study presents three key contributions: it is the first to achieve 360° multi-person gaze estimation under an upward-facing fisheye configuration; it introduces MPSGaze360, a large-scale synthetic dataset tailored for this task; and it validates the effectiveness of the proposed components, demonstrating the feasibility and potential of fisheye cameras for omnidirectional gaze estimation in multi-user scenarios.

Technology Category

• Computer Vision: Multi-modal Vision
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Machine Learning: Multimodal Learning

Application Category

• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Security and Privacy: Large-scale security measurements
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

We present GazeOnce360, a novel end-to-end model for multi-person gaze estimation from a single tabletop-mounted upward-facing fisheye camera. Unlike conventional approaches that rely on forward-facing cameras in constrained viewpoints, we address the underexplored setting of estimating the 3D gaze direction of multiple people distributed across a 360° scene from an upward fisheye perspective. To support research in this setting, we introduce MPSGaze360, a large-scale synthetic dataset rendered using Unreal Engine, featuring diverse multi-person configurations with accurate 3D gaze and eye landmark annotations. Our model tackles the severe distortion and perspective variation inherent in fisheye imagery by incorporating rotational convolutions and eye landmark supervision. To better capture fine-grained eye features crucial for gaze estimation, we propose a dual-resolution architecture that fuses global low-resolution context with high-resolution local eye regions. Experimental results demonstrate the effectiveness of each component in our model. This work highlights the feasibility and potential of fisheye-based 360° gaze estimation in practical multi-person scenarios. Project page: https://caizhuojiang.github.io/GazeOnce360/.

Problem

• gaze estimation
• fisheye camera
• 360-degree scene
• multi-person
• 3D gaze direction

Innovation

• fisheye gaze estimation
• multi-person 360° gaze
• global-local feature fusion
• rotational convolutions
• synthetic gaze dataset