Multiview Multi-Person Human Mesh Recovery Under Large Scenes with Occlusions

2026-07-27

AI Summary

This work addresses the challenge of accurately reconstructing 3D human meshes for multiple individuals under large-scale scenes, multi-view settings, and severe occlusions—conditions where existing methods often fail. To tackle this, the authors introduce MVMP-HMR, a large-scale synthetic dataset, and propose a novel model that fuses multi-view features into a scene-level 3D feature volume. Individual instance queries are derived from pelvis keypoints, and complete per-person meshes are decoded via a cross-attention mechanism. The approach further incorporates two novel loss functions: an occlusion-aware orientation loss and a 3D joint density loss, which jointly mitigate pose ambiguities caused by occlusion. Experiments demonstrate that the proposed method significantly outperforms current state-of-the-art approaches on a newly established, highly challenging benchmark, confirming its robustness and effectiveness in complex, large-scale scenarios with heavy occlusion.

Abstract

Human mesh recovery (HMR) aims to recover 3D human meshes from images. Most existing HMR benchmarks and methods focus on either multi-person reconstruction from a single view or single-person reconstruction from multiple views, where the number of subjects and the scene scale are relatively limited. Such settings are insufficient for real-world applications with large scenes and severe inter-person occlusions. To address this limitation, we introduce a large-scale synthetic benchmark for multiview multi-person HMR, termed MVMP-HMR. The proposed dataset contains 15 complex scenes with up to 50 camera views and 30 interacting persons, featuring large spatial coverage and severe occlusions, which significantly increases the difficulty of human mesh recovery. Based on this benchmark, we further propose a multiview multi-person whole-body human mesh recovery model, referred to as MVMP-HMR model. The model first fuses multiview features into a scene-level 3D feature volume, and then leverages pelvis joints predicted by a 3D pose estimation network to extract person-specific queries from the 3D feature volume. These human queries are cross-attended with the 3D feature volume and integrated to decode each person's 3D mesh. Moreover, we introduce two novel losses--the orientation loss and the 3D joint density loss--to alleviate orientation and pose ambiguities under severe occlusions. Experiments demonstrate that existing state-of-the-art HMR methods struggle on the proposed MVMP-HMR benchmark, while our method consistently outperforms prior SOTAs in large-scale scenes with severe occlusions.

Problem

• Human Mesh Recovery
• Multiview
• Multi-Person
• Occlusions
• Large Scenes

Innovation

• multiview multi-person human mesh recovery
• 3D feature volume
• cross-attention mechanism
• occlusion-robust losses
• synthetic benchmark