DecoyFace: Beyond Obfuscation via Controllable and Imperceptible Identity Misdirection for Privacy-Preserving Face Recognition

2026-07-19

AI Summary

This work addresses the vulnerability of intermediate features in split face recognition to feature inversion attacks by honest-but-curious (HBC) servers, where existing privacy-preserving methods often reveal their protective intent through reconstruction distortions, thereby inviting adaptive attacks. To counter this, the authors propose DecoyFace, a novel framework wherein the client decomposes intermediate representations into a reconstruction-sensitive subspace and its complementary counterpart. The former is infused with decoy identity cues to mislead unauthorized reconstructions, while the latter preserves genuine identity information for recognition. On the server side, an authorized normalization module suppresses the decoy components to recover valid identity representations. DecoyFace achieves, for the first time, imperceptible and controllable identity obfuscation, maintaining high recognition accuracy on LFW while reducing identity leakage rates to 2.93% and 0.74% under U-Net and Flow-Matching attacks, respectively, with reconstruction fidelity exceeding 99.78%.

Abstract

Split face recognition reduces client-side computation but exposes intermediate features to feature inversion attacks and unauthorized analysis by honest-but-curious (HBC) servers. Existing privacy-preserving face recognition methods mainly aim to resist unauthorized reconstruction, typically producing features whose inversion yields visibly degraded results, which may reveal the existence of protection and motivate adaptive attacks. To address this issue, we propose DecoyFace, an imperceptible decoy-oriented framework that steers unauthorized reconstruction toward a plausible but incorrect identity while preserving recognition utility. The key idea is to decompose the intermediate representation into a reconstruction-sensitive subspace and its complementary subspace. The client injects decoy identity cues into the reconstruction-sensitive subspace, while limited recognition-relevant evidence from the true sample is retained in the complementary subspace. On the server side, an authorized canonicalization module suppresses decoy-dominant components and recovers a recognition-friendly representation. This design addresses both attacker-side inversion from intercepted features and HBC server-side reconstruction from canonicalized representations. Experiments show that DecoyFace preserves competitive recognition accuracy while substantially reducing identity leakage to 2.93% under U-Net attacks and 0.74% under Flow-Matching attacks while yielding visually plausible and imperceptible reconstructions, with over 99.78% face validity on LFW dataset.

Problem

• face recognition
• privacy preservation
• feature inversion attack
• honest-but-curious server
• identity leakage

Innovation

• DecoyFace
• identity misdirection
• privacy-preserving face recognition
• feature inversion attack
• imperceptible decoy