TongueReenact: Geometry-Anchored Tongue Synthesis for Face Reenactment

2026-07-30

AI Summary

This work addresses the common omission of tongue dynamics in existing facial reenactment methods, which leads to inconsistent intra-oral motion. To overcome this limitation, we propose the first cross-identity tongue animation framework that leverages a geometry-driven representation and a spatially constrained latent-space masked diffusion model to synthesize realistic and boundary-smooth tongue motions. We introduce an innovative bootstrapped tongue segmentation training pipeline that requires no manual annotations and integrate a vision-language model to establish a large-scale perceptual evaluation protocol. Experimental results demonstrate that our method outperforms current baselines by more than twofold across all tongue-related metrics, significantly enhancing both the realism and anatomical consistency of tongue motion in facial reenactment.

Abstract

Modern face reenactment systems achieve impressive pose and expression transfer using geometry-driven representations. However, they largely ignore tongue dynamics, leading to anatomically inconsistent mouth interiors during speech and expressive motions. We introduce the first framework for cross-identity tongue dynamics transfer in face reenactment. We propose a foundation-model-assisted bootstrapping pipeline that produces a dedicated tongue segmentation model for in-the-wild reenactment without curated annotations. We further introduce a spatially constrained latent masked diffusion model for realistic tongue synthesis, with adaptive mask dilation for seamless mouth boundary transitions. Extensive experiments demonstrate improvements of more than two times over all baselines on every tongue-specific metric. We additionally propose a VLM-based evaluation protocol that replicates expert annotation at scale, confirming perceptual superiority across all ablation variants.

Problem

• tongue dynamics
• face reenactment
• anatomical consistency
• mouth interior
• speech animation

Innovation

• tongue dynamics transfer
• geometry-anchored synthesis
• masked diffusion model
• foundation-model-assisted bootstrapping
• VLM-based evaluation