DiffEyeSyn: Diffusion-based User-specific Eye Movement Synthesis

2024-09-02

arXiv.org

AI Summary

Existing eye movement modeling approaches primarily focus on low-frequency features, neglecting user-specific fine-grained motion patterns embedded in high-frequency components (>30 Hz), thereby failing to generate identity-discriminative gaze sequences. This work introduces the first user-specific synthesis framework for high-frequency eye movements: it models individual high-frequency gaze characteristics as injectable “identity noise” and proposes a user-identity-guided conditional diffusion model. To jointly optimize identity discriminability and motion naturalness, we incorporate pretrained authentication embeddings and a spatial-domain identity fidelity loss. Evaluated on two public high-frequency eye-tracking datasets, our synthesized sequences are perceptually indistinguishable from real recordings. The framework successfully supports diverse downstream applications—including gaze imputation, super-resolution, animation driving, biometric identification, and context-aware modeling—demonstrating both strong identity preservation and dynamic plausibility.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• User Modeling, Personalization and Recommendation: User privacy protection in personalized systems
• Security and Privacy: Large-scale security measurements
• Responsible Web: Data and user privacy-enhancing technologies for the Web

Abstract

High-frequency components in eye gaze data contain user-specific information promising for various applications, but existing gaze modelling methods focus on low frequencies of typically not more than 30 Hz. We present DiffEyeSyn -- the first computational method to synthesise high-frequency gaze data, including eye movement characteristics specific to individual users. The key idea is to consider the high-frequency, user-specific information as a special type of noise in eye movement data. This perspective reshapes eye movement synthesis into the task of injecting this user-specific noise into any given eye movement sequence. We formulate this injection task as a conditional diffusion process in which the synthesis is conditioned on user-specific embeddings extracted from the gaze data using pre-trained models for user authentication. We propose user identity guidance -- a novel loss function that allows our model to preserve user identity while generating human-like eye movements in the spatial domain. Experiment results on two public high-frequency eye movement biometric datasets show that our synthetic eye movements are indistinguishable from real human eye movements. Furthermore, we demonstrate that DiffEyeSyn can be used to synthesise eye gaze data at scale and for different downstream tasks, such as gaze data imputation and gaze data super-resolution. As such, our work lays the methodological foundations for personalised eye movement synthesis that has significant application potential, such as for character animation, eye movement biometrics, or gaze-based activity and context recognition.

Problem

• Synthesize user-specific high-frequency eye movements
• Model user-specific noise in eye movement data
• Generate realistic gaze data for downstream tasks

Innovation

• Uses conditional diffusion for eye movement synthesis
• Injects user-specific noise as identity embeddings
• Proposes user identity guidance loss function