GRAPE: Graduated Routing for Articulated Portrait mesh Estimation

2026-07-26

AI Summary

This work addresses the challenges of high-fidelity 3D human portrait reconstruction from monocular images, particularly the coupling between head and torso motion and the difficulty in disentangling jaw articulation from facial expressions. To tackle these issues, the authors propose the Portrait Parametric Model (PPM), which explicitly models the kinematic chain from torso to head and integrates the canonical spaces of FLAME and SMPL-X. They further introduce a Progressive Anatomical Alignment (PAA) network featuring a graduated-mask router—a coarse-to-fine expert routing mechanism guided by anatomical priors—and multi-source supervision from sparse keypoints, feature distillation, foreground masks, and relative geometric constraints. The method significantly outperforms existing approaches in reconstruction fidelity, pose alignment accuracy, and jaw-expression disentanglement, leading to notable improvements in downstream tasks such as speech-driven talking-head animation and 3D portrait generation.

Abstract

Articulated portrait mesh estimation is fundamental to 3D understanding, avatar generation, and immersive interaction. Existing approaches primarily rely on 3D Morphable Models (3DMMs). However, face-centric models suffer from the "floating head" assumption, conflating head pose with global rotation due to the lack of neck kinematics. Conversely, body-centric models lack high-fidelity facial expression capabilities. Furthermore, current methods struggle to disentangle jaw articulation from expression blendshapes, often over-relying on expressions for mouth opening. These limitations make monocular portrait recovery difficult across representation, supervision, and anatomical parameter estimation. To address these limitations, we introduce GRAPE(Graduated Routing for Articulated Portrait mesh Estimation). We build a Portrait Parametric Model (PPM) with an explicit torso-to-head kinematic chain and a canonical injection step to merge FLAME and the SMPL-X torso. We propose a Progressive Anatomical Alignment (PAA) network, which is composed of a pretrained portrait encoder, a Graduated-Mask Router, and coarse-to-fine experts that follow the portrait anatomical prior. We then train this network with multi-source supervision that combines sparse anatomical keypoints, feature distillation, foreground mask constraints, and relative geometry constraints. Experiments show that GRAPE improves portrait mesh recovery quality, pose alignment, and jaw--expression disentanglement over prior methods. We also demonstrate that our method can benefit the downstream tasks of audio-driven talking-head generation and 3D portrait generation.

Problem

• articulated portrait mesh estimation
• 3D Morphable Models
• jaw-expression disentanglement
• neck kinematics
• monocular portrait recovery

Innovation

• articulated portrait mesh
• kinematic chain
• Progressive Anatomical Alignment
• jaw-expression disentanglement
• multi-source supervision