Cascaded Dual Vision Transformer for Accurate Facial Landmark Detection

2024-11-08

IEEE Workshop/Winter Conference on Applications of Computer Vision

AI Summary

This work addresses two key challenges in facial landmark detection: insufficient geometric relationship modeling and degradation of low-level features. To this end, we propose a cascaded dual-branch Vision Transformer (ViT) architecture. The architecture jointly integrates a channel-partitioned ViT—capturing inter-channel semantic dependencies—and a spatial-partitioned ViT—modeling long-range geometric correlations via linear bases in heatmaps—augmented with long-range skip connections to explicitly preserve low-level image features and mitigate information attenuation induced by intermediate supervision. Adopting a heatmap regression paradigm with cascaded prediction, our method enhances localization accuracy. Extensive experiments demonstrate state-of-the-art performance on three major benchmarks—WFLW, COFW, and 300W—with significantly reduced mean errors. Results validate the effectiveness of the dual-path ViT collaboration and the feature fidelity mechanism.

Technology Category

• Computer Vision: Multi-modal Vision
• Natural Language Processing: Fact-Checking / Misinformation Detection (NLP Focus)
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Vertical and domain-specific search

Abstract

Facial landmark detection is a fundamental problem in computer vision for many downstream applications. This paper introduces a new facial landmark detector based on vision transformers, which consists of two unique designs: Dual Vision Transformer (D-ViT) and Long Skip Connections (LSC). Based on the observation that the channel dimension of feature maps essentially represents the linear bases of the heatmap space, we propose learning the inter-connections between these linear bases to model the inherent geometric relations among landmarks via channel-split ViT. We integrate such channel-split ViT into the standard vision transformer (i.e., spatial-split ViT),forming our Dual Vision Transformer to constitute the prediction blocks. We also suggest using long skip connections to deliver low-level image features to all prediction blocks, thereby preventing useful information from being discarded by intermediate supervision. Extensive experiments are conducted to evaluate the performance of our proposal on the widely used benchmarks, i.e., WFLW [45], COFW [3], and 300W [34], demonstrating that our model outperforms the previous SOTAs across all three benchmarks.

Problem

• Improves facial landmark detection accuracy using vision transformers
• Models geometric relations among landmarks via Channel-split ViT
• Prevents information loss with long skip connections in prediction blocks

Innovation

• Dual Vision Transformer combines channel-split and spatial-split ViT
• Long skip connections preserve low-level image features
• Channel-split ViT models landmark geometric relations via interconnections