EyeTrAES: Fine-grained, Low-Latency Eye Tracking via Adaptive Event Slicing

2024-09-27

Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies

AI Summary

To address the limitations of conventional RGB cameras—low temporal resolution, high computational overhead, and difficulty capturing rapid pupil dynamics—in eye-tracking applications, this paper proposes a lightweight pupil tracking and identity authentication framework leveraging neuromorphic event cameras. Methodologically: (1) an adaptive event slicing algorithm dynamically accumulates asynchronous event streams to generate monocular event frames; (2) an end-to-end lightweight segmentation and tracking pipeline is developed; (3) short-term pupil kinematics—namely position, velocity, and acceleration—are first modeled as individual-specific biometric traits, with identity authentication performed efficiently via random forest classification. Experiments demonstrate a pupil tracking IoU of 92% (a >6% improvement), over threefold reduction in end-to-end latency, an authentication accuracy of 82%, and inference time of only 12 ms per frame.

Technology Category

• Computer Vision: Motion & Tracking
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Machine Learning: Hardware-aware ML

Application Category

• Security and Privacy: Tracking, profiling, and countermeasures against them
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Eye-tracking technology has gained significant attention in recent years due to its wide range of applications in human-computer interaction, virtual and augmented reality, and wearable health. Traditional RGB camera-based eye-tracking systems often struggle with poor temporal resolution and computational constraints, limiting their effectiveness in capturing rapid eye movements. To address these limitations, we propose EyeTrAES, a novel approach using neuromorphic event cameras for high-fidelity tracking of natural pupillary movement that shows significant kinematic variance. One of EyeTrAES's highlights is the use of a novel adaptive windowing/slicing algorithm that ensures just the right amount of descriptive asynchronous event data accumulation within an event frame, across a wide range of eye movement patterns. EyeTrAES then applies lightweight image processing functions over accumulated event frames from just a single eye to perform pupil segmentation and tracking (as opposed to gaze-based techniques that require simultaneous tracking of both eyes). We show that these two techniques boost pupil tracking fidelity by 6+%, achieving IoU~=92%, while incurring at least 3x lower latency than competing pure event-based eye tracking alternatives [38]. We additionally demonstrate that the microscopic pupillary motion captured by EyeTrAES exhibits distinctive variations across individuals and can thus serve as a biometric fingerprint. For robust user authentication, we train a lightweight per-user Random Forest classifier using a novel feature vector of short-term pupillary kinematics, comprising a sliding window of pupil (location, velocity, acceleration) triples. Experimental studies with two different datasets (capturing eye movement across a range of environmental contexts) demonstrate that the EyeTrAES-based authentication technique can simultaneously achieve high authentication accuracy (~=0.82) and low processing latency (~=12ms), and significantly outperform multiple state-of-the-art competitive baselines.

Problem

• Improving temporal resolution in eye-tracking systems
• Reducing latency in event-based eye tracking
• Enhancing biometric authentication via pupillary motion

Innovation

• Adaptive event slicing for precise eye tracking
• Lightweight image processing for pupil segmentation
• Random Forest classifier for biometric authentication