DisPOSE: Projected Polystochastic Diffusion for Self-Supervised Multi-View 3D Human Pose Estimation

2026-06-05

AI Summary

This work addresses the limited generalization and challenging view-to-person assignment inherent in self-supervised multi-view multi-person 3D human pose estimation by introducing a novel framework based on a multi-random tensor diffusion mechanism. The proposed method formulates the discrete person-view assignment as a differentiable generative diffusion process, integrating differentiable Sinkhorn projection with a hypergraph convolutional decoder to enable self-supervised 3D pose reconstruction from 2D image priors. By innovatively decoupling the assignment problem from root joint regression, the approach significantly enhances robustness to camera configurations and improves label efficiency. It outperforms existing self-supervised methods on standard benchmarks and maintains 99% of its performance on a new, highly occluded operating room dataset using only 10% pseudo-labels.

Abstract

Recovering 3D human poses for multiple individuals from different camera views is a fundamental bottleneck for analyzing interacting behaviors. Existing self-supervised approaches leverage synthetic catalogues of 3D poses; however, this leads to poor generalization in real-world scenarios due to distribution shifts. We therefore introduce DisPOSE, a self-supervised framework that approximates the inherently discrete multi-view person-assignment problem as a generative diffusion process over the space of polystochastic tensors. By employing differentiable Sinkhorn projections during denoising, our model learns to guide solutions toward valid and feasible assignments based on 2D image priors. The complete 3D skeletons of localized individuals are then regressed using a Hypergraph-Convolutional Decoder that explicitly models relational structures and articulated joints across multiple views. The proposed approach outperforms current state-of-the-art self-supervised methods on standard datasets and demonstrates strong performance on a newly proposed benchmark featuring highly occluded scenes from surgical operating rooms. Our diffusion-based localization demonstrates high label efficiency, retaining 99% of its performance with only 10% of the pseudo-labels. Notably, disentangling the assignment and root regression components while maintaining differentiability makes DisPOSE nearly agnostic to different camera arrangements.

Problem

• multi-view 3D human pose estimation
• self-supervised learning
• person assignment
• distribution shift
• occluded scenes

Innovation

• polystochastic diffusion
• self-supervised 3D pose estimation
• differentiable Sinkhorn projection
• hypergraph-convolutional decoder
• multi-view person assignment