CodeFormer++: Blind Face Restoration Using Deformable Registration and Deep Metric Learning

2025-10-05

AI Summary

Blind face restoration (BFR) faces a fundamental trade-off between visual quality and identity fidelity. To address this, we propose a decoupled framework that decomposes BFR into three coordinated subtasks: identity preservation, high-fidelity generation, and dynamic fusion. Our key contributions are: (1) a learnable deformable face registration module that achieves semantic alignment between the degraded input and the generated face; (2) a deep metric learning–based feature fusion mechanism that constructs discriminative positive/negative sample pairs to enhance identity consistency; and (3) a texture-guided restoration network that dynamically transfers generative priors to recover fine-grained details. The model is trained end-to-end on both synthetic and real-world data. Extensive experiments demonstrate that our method significantly outperforms state-of-the-art approaches across multiple benchmarks, achieving simultaneous improvements in perceptual quality and identity preservation.

Technology Category

Computer Vision: Generative Adversarial Networks (GANs) for VisionMachine Learning: Multimodal LearningIntelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for rankingUser Modeling, Personalization and Recommendation: Fairness-aware retrieval and rankingResponsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

Blind face restoration (BFR) has attracted increasing attention with the rise of generative methods. Most existing approaches integrate generative priors into the restoration pro- cess, aiming to jointly address facial detail generation and identity preservation. However, these methods often suffer from a trade-off between visual quality and identity fidelity, leading to either identity distortion or suboptimal degradation removal. In this paper, we present CodeFormer++, a novel framework that maximizes the utility of generative priors for high-quality face restoration while preserving identity. We decompose BFR into three sub-tasks: (i) identity- preserving face restoration, (ii) high-quality face generation, and (iii) dynamic fusion of identity features with realistic texture details. Our method makes three key contributions: (1) a learning-based deformable face registration module that semantically aligns generated and restored faces; (2) a texture guided restoration network to dynamically extract and transfer the texture of generated face to boost the quality of identity-preserving restored face; and (3) the integration of deep metric learning for BFR with the generation of informative positive and hard negative samples to better fuse identity- preserving and generative features. Extensive experiments on real-world and synthetic datasets demonstrate that, the pro- posed CodeFormer++ achieves superior performance in terms of both visual fidelity and identity consistency.

Problem

• Balancing visual quality and identity preservation in face restoration
• Semantically aligning generated and restored faces using deformable registration
• Integrating deep metric learning to fuse identity and generative features

Innovation

• Learning-based deformable registration aligns faces semantically
• Texture guided network transfers generated face details
• Deep metric learning fuses identity and generative features