How Private is Private? A Comparative Study for Face De-Identification

2026-10-07

AI Summary

This study addresses the fragmented evaluation criteria and difficulty of cross-method comparison in face anonymization by proposing a hierarchical, unified evaluation paradigm. We construct the UtilFace benchmark alongside the HiFD hierarchical metric, establishing a three-level utility system with a configurable scoring mechanism. Through consistent computation via pretrained estimators and weighted harmonic mean aggregation, this framework enables unified quantitative assessment of both identity suppression and utility preservation. The research systematically reveals performance trade-offs and failure modes in existing methods. Furthermore, the complete benchmark and toolkit are open-sourced, providing a standardized evaluation foundation for the field and significantly facilitating reproducible research.

Abstract

Face de-identification (FDeID) has emerged as a critical privacy-preserving technology, yet its evaluation remains fundamentally fragmented. Existing protocols rely on inconsistent metrics, heterogeneous datasets, and partial annotation coverage, so methods targeting different utility dimensions, such as landmark versus expression preservation, are reported on different benchmarks under different metrics, rendering cross-method comparison infeasible. We revisit FDeID evaluation from both the data and metric perspectives. On the data side, we introduce UtilFace, a curated, demographically balanced benchmark with high identity diversity, assembled from four large-scale face datasets through identity-aware cleaning, resolution enhancement, and stratified filtering. On the metric side, we propose HiFD, a Hierarchical Face De-identification metric that unifies identity suppression, multi-level utility preservation, and image quality under a single consistency-based paradigm: every component is computed from pretrained estimators' outputs on the original face and its de-identified counterpart, directly quantifying how much identity is suppressed and how much downstream-perceivable utility survives. HiFD organizes facial signals into a three-level utility hierarchy spanning macro cues (L1), micro cues (L2), and imperceptible cues (L3), and aggregates the five resulting components into a single interpretable score via weighted harmonic mean, with configurable application-specific profiles. Using this unified protocol, we conduct a comprehensive comparative study spanning adversarial, GAN-based, and diffusion-based methods, surfacing trade-offs and failure modes that remain invisible under existing protocols. We release the benchmark and evaluation toolkit to foster systematic and reproducible research in privacy-preserving human face analysis.

Problem

• Face De-Identification
• Privacy Preservation
• Evaluation Benchmark
• Cross-method Comparison

Innovation

• Face De-identification
• UtilFace Benchmark
• Hierarchical Metric (HiFD)
• Utility Preservation
• Privacy Evaluation