Human Pose-Constrained UV Map Estimation

2025-01-15

AI Summary

Existing UV mapping methods operate pixel-wise, lacking global anatomical consistency and thus producing implausible or locally inaccurate mappings. This work proposes Pose-Constrained Continuous Surface Embeddings (PC-CSE), the first approach to explicitly incorporate real-time estimated full-body pose as a strong geometric constraint into UV mapping. PC-CSE achieves this via continuous surface embedding, pose-guided pixel-to-vertex correspondence, and multi-scale feature alignment—jointly ensuring global coherence and local fidelity. Trained and evaluated end-to-end on DensePose COCO, PC-CSE significantly reduces invalid mapping rates and improves anatomical plausibility and structural consistency of UV maps. Experiments demonstrate that full-body pose constraints outperform upper-body-only constraints and that PC-CSE is compatible with arbitrary 2D pose estimators. Furthermore, our analysis uncovers systematic annotation inconsistencies in the DensePose dataset—highlighting a previously underreported limitation in current benchmarks.

Technology Category

• Computer Vision: Biometrics, Face, Gesture & Pose
• Machine Learning: Learning with Manifolds
• Constraint Satisfaction and Optimization: Distributed CSP/Optimization

Application Category

• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

UV map estimation is used in computer vision for detailed analysis of human posture or activity. Previous methods assign pixels to body model vertices by comparing pixel descriptors independently, without enforcing global coherence or plausibility in the UV map. We propose Pose-Constrained Continuous Surface Embeddings (PC-CSE), which integrates estimated 2D human pose into the pixel-to-vertex assignment process. The pose provides global anatomical constraints, ensuring that UV maps remain coherent while preserving local precision. Evaluation on DensePose COCO demonstrates consistent improvement, regardless of the chosen 2D human pose model. Whole-body poses offer better constraints by incorporating additional details about the hands and feet. Conditioning UV maps with human pose reduces invalid mappings and enhances anatomical plausibility. In addition, we highlight inconsistencies in the ground-truth annotations.

Problem

• UV Mapping
• Human Pose Estimation
• Computer Vision

Innovation

• Pose-Constrained Continuous Surface Embedding
• UV maps estimation
• DensePose COCO dataset