AmbientEye: A Dataset for Pupil Segmentation under Natural Ambient Infrared Illumination

2026-06-02

AI Summary

This study addresses the challenge of deploying eye-tracking systems in unconstrained outdoor environments, where reliance on active infrared illumination renders existing solutions impractical under natural lighting conditions. To explore the feasibility of robust pupil segmentation using only ambient sunlight and passive infrared cameras, we introduce AmbientEye—the first large-scale dataset for pupil segmentation under natural infrared illumination. AmbientEye encompasses diverse participants from multiple countries, varied solar angles, and heterogeneous camera configurations, with annotations generated via SAM2-assisted automatic segmentation followed by meticulous manual refinement. Experimental results reveal a significant performance gap between models trained on AmbientEye (IoU = 0.767) and those operating in controlled settings (IoU = 0.928), underscoring the inherent difficulty of this scenario and establishing the first benchmark for gaze tracking without active illumination.

Abstract

Eye tracking is essential for smart glasses, as it provides insight into user attention for ambient intelligence applications. However, most existing eye-tracking systems rely on active infrared (IR) illumination, creating practical barriers to all-day outdoor use due to power consumption. In this paper, we investigate whether passive IR cameras alone, without any active IR light source, can enable reliable pupil detection in unconstrained outdoor environments, where ambient sunlight serves as the sole illumination source. To support this investigation, we introduce AmbientEye, a large-scale dataset of 2,606,225 eye images collected from 35 participants from 19 countries. It is captured outdoors under natural sunlight with two off-axis camera configurations and two sun-orientation conditions. We provide high-quality pupil annotation through SAM2 automatic segmentation, followed by refinement by human annotators. We benchmark a state-of-the-art pupil segmentation algorithm on our dataset and compare its performance with that on existing datasets under controlled IR illumination. Results reveal a substantial drop in pupil segmentation performance from 0.928 on controlled IR datasets to 0.767 on AmbientEye. This performance gap highlights the challenge of the ambient-light setting. This positions AmbientEye as a first benchmark for an unexplored and highly practical eye-tracking scenario.

Problem

• pupil segmentation
• ambient infrared illumination
• eye tracking
• outdoor environment
• passive IR camera

Innovation

• passive infrared eye tracking
• pupil segmentation
• natural ambient illumination
• outdoor eye dataset
• smart glasses