Motion-Based User Identification across XR and Metaverse Applications by Deep Classification and Similarity Learning

2025-09-10

AI Summary

This study investigates the cross-application generalization capability of motion-based user identification models in extended reality (XR) environments. Addressing the limited generalizability and unclear privacy implications of existing classification and similarity-learning approaches, we introduce the first benchmark dataset encompassing five distinct XR application scenarios, with multi-task motion behavioral data from 49 participants—publicly released alongside source code. Experimental results show that deep classification and metric learning models achieve high intra-scenario identification accuracy (>95%), yet suffer substantial performance degradation when evaluated cross-scenario (average drop of 32.7%). This reveals a critical generalization bottleneck for biometric authentication in metaverse-scale XR systems, alongside non-negligible privacy leakage risks. Our work establishes the first systematic, cross-scenario benchmark and empirical evaluation framework for XR-based identity authentication.

Technology Category

• Computer Vision: Biometrics, Face, Gesture & Pose
• Machine Learning: Privacy
• Humans and AI: Game Design — Virtual Humans, NPCs and Autonomous Characters

Application Category

• User Modeling, Personalization and Recommendation: User privacy protection in personalized systems
• Security and Privacy: Large-scale security measurements
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics

Abstract

This paper examines the generalization capacity of two state-of-the-art classification and similarity learning models in reliably identifying users based on their motions in various Extended Reality (XR) applications. We developed a novel dataset containing a wide range of motion data from 49 users in five different XR applications: four XR games with distinct tasks and action patterns, and an additional social XR application with no predefined task sets. The dataset is used to evaluate the performance and, in particular, the generalization capacity of the two models across applications. Our results indicate that while the models can accurately identify individuals within the same application, their ability to identify users across different XR applications remains limited. Overall, our results provide insight into current models generalization capabilities and suitability as biometric methods for user verification and identification. The results also serve as a much-needed risk assessment of hazardous and unwanted user identification in XR and Metaverse applications. Our cross-application XR motion dataset and code are made available to the public to encourage similar research on the generalization of motion-based user identification in typical Metaverse application use cases.

Problem

• Evaluating motion-based user identification generalization across XR applications
• Assessing biometric model performance for cross-application user verification
• Investigating risks of unwanted user identification in Metaverse environments

Innovation

• Deep classification and similarity learning models
• Cross-application XR motion dataset development
• Generalization capacity evaluation across XR applications