GazeSCRNN: Event-based Near-eye Gaze Tracking using a Spiking Neural Network

2025-03-20

AI Summary

To address the insufficient accuracy and real-time performance of conventional eye-tracking systems in dynamic scenarios, this paper proposes the first spiking neural network (SNN)-based near-eye gaze estimation method leveraging event cameras. Methodologically, it introduces an adaptive leaky integrate-and-fire (ALIF) neuron model integrated with a hybrid convolutional-recurrent architecture, coupled with dynamic event framing and forward-propagation-through-time (FPTT) training to effectively model sparse, asynchronous event streams. Evaluated on the EV-Eye dataset, the method achieves a mean angular error of 6.034° and a pupil localization error of 2.094 mm—substantially outperforming existing event-driven approaches. These results demonstrate the feasibility and superiority of SNNs for low-latency, low-power dynamic eye-tracking, establishing a novel paradigm for edge-deployable real-time gaze estimation.

Technology Category

• Cognitive Modeling & Cognitive Systems: Neural Spike Coding
• Computer Vision: Motion & Tracking
• Machine Learning: Learning on the Edge & Model Compression

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Security and Privacy: Large-scale security measurements
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

This work introduces GazeSCRNN, a novel spiking convolutional recurrent neural network designed for event-based near-eye gaze tracking. Leveraging the high temporal resolution, energy efficiency, and compatibility of Dynamic Vision Sensor (DVS) cameras with event-based systems, GazeSCRNN uses a spiking neural network (SNN) to address the limitations of traditional gaze-tracking systems in capturing dynamic movements. The proposed model processes event streams from DVS cameras using Adaptive Leaky-Integrate-and-Fire (ALIF) neurons and a hybrid architecture optimized for spatio-temporal data. Extensive evaluations on the EV-Eye dataset demonstrate the model's accuracy in predicting gaze vectors. In addition, we conducted ablation studies to reveal the importance of the ALIF neurons, dynamic event framing, and training techniques, such as Forward-Propagation-Through-Time, in enhancing overall system performance. The most accurate model achieved a Mean Angle Error (MAE) of 6.034{deg} and a Mean Pupil Error (MPE) of 2.094 mm. Consequently, this work is pioneering in demonstrating the feasibility of using SNNs for event-based gaze tracking, while shedding light on critical challenges and opportunities for further improvement.

Problem

• Develops GazeSCRNN for event-based near-eye gaze tracking
• Addresses limitations of traditional gaze-tracking systems
• Demonstrates feasibility of spiking neural networks in gaze tracking

Innovation

• Spiking neural network for gaze tracking
• Dynamic Vision Sensor camera integration
• Adaptive Leaky-Integrate-and-Fire neurons