Emergent Region-Level Facial Correspondence in Frozen Vision Foundation Models

2026-07-15

AI Summary

This study investigates whether frozen self-supervised vision models can establish cross-identity, cross-temporal semantic correspondences at the facial region level without any face-specific training. Leveraging a frozen DINOv2 ViT-L/16 model to extract patch embeddings and utilizing facial part annotations from FaRL, the authors evaluate performance through cross-identity nearest-neighbor matching and temporal label propagation. They demonstrate for the first time that general-purpose vision models, without fine-tuning, can construct a facial region coordinate system capable of accurately distinguishing anatomical structures such as eyes, nose, and mouth, with features from an intermediate layer (block 18) exhibiting the highest discriminative power. On CelebDF-v2, the approach achieves 83.0% region semantic accuracy (versus a random baseline of 23.0%) and 95.5% temporal tracking accuracy, representing a 4.93× improvement over the baseline in region discrimination.

Abstract

Frozen self-supervised vision models can align parts of generic objects, but it remains unclear whether this correspondence extends to human faces, where global layout is shared while identity-specific appearance varies sharply. We test whether frozen DINOv3 features define a region-level facial coordinate system: a feature space in which eyes, brows, nose, mouth, skin, and hair remain distinguishable across people and across time without face-specific training. Using DINOv3 ViT-L/16 patch embeddings and FaRL only as a face-part labeling interface, we evaluate cross-identity nearest-neighbor matching and temporal label propagation on 200 CelebDF-v2 real videos. DINOv3 achieves 83.0% region-level semantic accuracy under unconstrained cross-identity matching, compared with a 23.0% area-weighted random baseline, and 95.5% temporal tracking accuracy without a learned temporal module. A no-FaRL control collapses to 0.9%, showing that FaRL supplies semantic initialization while DINOv3 supplies dense spatial correspondence. The strongest correspondence appears at an intermediate layer: block 18 gives a 4.93x same-region versus cross-region discrimination ratio, compared with 1.48x at the final block. Against CLIP ViT-L/14, DINOv3 shows only a small aggregate advantage but a +16.8 pp gain on anatomical regions, indicating that image-level contrastive supervision captures coarse facial layout but not fine-grained anatomical identity. These results establish frozen DINOv3 as a strong zero-shot representation for region-level facial correspondence and identify intermediate self-supervised features as the most useful layer for dense face analysis.

Problem

• facial correspondence
• frozen vision models
• region-level alignment
• self-supervised learning
• face representation

Innovation

• frozen vision models
• region-level facial correspondence
• self-supervised learning
• DINOv3
• zero-shot face analysis