Pose-Robust Calibration Strategy for Point-of-Gaze Estimation on Mobile Phones

2025-08-13

AI Summary

Appearance-based point-of-gaze (PoG) estimation on mobile devices faces two key challenges: poor inter-subject generalization necessitating subject-specific calibration, and post-calibration model sensitivity to head pose variations. To address these, we propose a robust dynamic calibration strategy wherein users naturally move their mobile device while fixating on static calibration points—thereby actively capturing multi-pose facial images and enhancing pose-invariance in the learned model. We introduce MobilePoG, the first multi-pose PoG benchmark tailored for mobile scenarios, and systematically analyze the impact of calibration point distribution and pose diversity on estimation accuracy. Experiments demonstrate that our method significantly reduces pose-induced estimation errors, outperforming conventional static calibration in real-world settings. The approach achieves high efficiency, strong user-friendliness, and superior cross-subject generalization without requiring explicit pose normalization or additional hardware.

Technology Category

• Computer Vision: Biometrics, Face, Gesture & Pose
• Intelligent Robots: State Estimation
• Machine Learning: Calibration & Uncertainty Quantification

Application Category

• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Although appearance-based point-of-gaze (PoG) estimation has improved, the estimators still struggle to generalize across individuals due to personal differences. Therefore, person-specific calibration is required for accurate PoG estimation. However, calibrated PoG estimators are often sensitive to head pose variations. To address this, we investigate the key factors influencing calibrated estimators and explore pose-robust calibration strategies. Specifically, we first construct a benchmark, MobilePoG, which includes facial images from 32 individuals focusing on designated points under either fixed or continuously changing head poses. Using this benchmark, we systematically analyze how the diversity of calibration points and head poses influences estimation accuracy. Our experiments show that introducing a wider range of head poses during calibration improves the estimator's ability to handle pose variation. Building on this insight, we propose a dynamic calibration strategy in which users fixate on calibration points while moving their phones. This strategy naturally introduces head pose variation during a user-friendly and efficient calibration process, ultimately producing a better calibrated PoG estimator that is less sensitive to head pose variations than those using conventional calibration strategies. Codes and datasets are available at our project page.

Problem

• Improving point-of-gaze estimation accuracy across individuals
• Reducing sensitivity to head pose variations in calibration
• Developing pose-robust calibration strategies for mobile phones

Innovation

• Dynamic calibration strategy introducing head pose variation
• User moves phone during fixation on calibration points
• Benchmark MobilePoG with 32 individuals' facial images