Towards Privacy-preserving Photorealistic Self-avatars in Mixed Reality

2025-07-29

AI Summary

This study addresses the privacy–realism trade-off in generating 3D virtual avatars for mixed reality. We propose a generative framework that jointly preserves demographic identity fidelity (e.g., age, gender, race) and ensures facial biometric de-identification. Methodologically, we introduce two novel identity obfuscation techniques operating in the latent space of an identity-encoding model: (1) a differential privacy–based feature perturbation mechanism, and (2) a parameterizable, fine-grained identity shift strategy. Both are integrated into a unified 2D/3D end-to-end de-identification synthesis architecture. Experiments demonstrate that our approach achieves optimal balance among visual realism, demographic attribute consistency, and irreversible facial biometric unlinkability. The resulting avatars significantly mitigate risks of identity misuse and targeted adversarial attacks, making them suitable for public, social mixed-reality applications.

Technology Category

• Machine Learning: Privacy
• Computer Vision: Bias, Fairness & Privacy
• Humans and AI: Game Design — Virtual Humans, NPCs and Autonomous Characters

Application Category

• User Modeling, Personalization and Recommendation: User privacy protection in personalized systems
• Security and Privacy: Data transparency and provenance
• Economics, Online Markets and Human Computation: Fairness, privacy, and diversity in economic environments

Abstract

Photorealistic 3D avatar generation has rapidly improved in recent years, and realistic avatars that match a user's true appearance are more feasible in Mixed Reality (MR) than ever before. Yet, there are known risks to sharing one's likeness online, and photorealistic MR avatars could exacerbate these risks. If user likenesses were to be shared broadly, there are risks for cyber abuse or targeted fraud based on user appearances. We propose an alternate avatar rendering scheme for broader social MR -- synthesizing realistic avatars that preserve a user's demographic identity while being distinct enough from the individual user to protect facial biometric information. We introduce a methodology for privatizing appearance by isolating identity within the feature space of identity-encoding generative models. We develop two algorithms that then obfuscate identity: epsmethod{} provides differential privacy guarantees and hetamethod{} provides fine-grained control for the level of identity offset. These methods are shown to successfully generate de-identified virtual avatars across multiple generative architectures in 2D and 3D. With these techniques, it is possible to protect user privacy while largely preserving attributes related to sense of self. Employing these techniques in public settings could enable the use of photorealistic avatars broadly in MR, maintaining high realism and immersion without privacy risk.

Problem

• Protect user privacy in photorealistic MR avatars
• Prevent cyber abuse and fraud via likeness sharing
• Balance demographic identity preservation with facial biometric protection

Innovation

• Differential privacy guarantees for avatars
• Fine-grained control of identity offset
• Isolating identity in generative model features