RapidPoseTriangulation: Multi-view Multi-person Whole-body Human Pose Triangulation in a Millisecond

2025-03-27

AI Summary

This paper addresses the challenge of real-time, high-precision estimation of full-body 3D poses—including facial expressions and finger articulation—under high-speed, multi-view conditions. We propose the first sub-millisecond triangulation framework for holistic body pose estimation. Methodologically, it integrates geometrically constrained fast SVD-based triangulation, differentiable pose priors, multi-scale view-aware feature alignment, and end-to-end joint optimization, yielding a lightweight and robust multi-view consistency architecture. Key contributions include: (i) the first triangulation pipeline achieving <1 ms per frame for full-body pose estimation; and (ii) significantly improved generalization across datasets and configurations (e.g., varying numbers of views or camera calibrations). Evaluated on CMU Panoptic and TotalCapture benchmarks, our method achieves state-of-the-art accuracy (MPJPE < 45 mm) with an average runtime of only 0.8 ms per frame, enabling real-time processing with over 16 synchronized views.

Technology Category

• Computer Vision: Biometrics, Face, Gesture & Pose
• Intelligent Robots: State Estimation
• Machine Learning: Multi-instance/Multi-view Learning

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Security and Privacy: Large-scale security measurements
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation

Abstract

The integration of multi-view imaging and pose estimation represents a significant advance in computer vision applications, offering new possibilities for understanding human movement and interactions. This work presents a new algorithm that improves multi-view multi-person pose estimation, focusing on fast triangulation speeds and good generalization capabilities. The approach extends to whole-body pose estimation, capturing details from facial expressions to finger movements across multiple individuals and viewpoints. Adaptability to different settings is demonstrated through strong performance across unseen datasets and configurations. To support further progress in this field, all of this work is publicly accessible.

Problem

• Fast multi-view multi-person pose triangulation
• Whole-body pose estimation across viewpoints
• Generalization to unseen datasets and settings

Innovation

• Fast multi-view multi-person pose triangulation
• Whole-body pose estimation with detailed capture
• Strong generalization across unseen datasets