Few-Shot Multi-Human Neural Rendering Using Geometry Constraints

2025-02-11

AI Summary

This paper addresses the challenging problem of joint geometric and appearance reconstruction for multi-person scenes from extremely sparse image inputs (only 3–5 frames), where severe inter-person occlusion, geometric ambiguity, and lighting variations severely degrade modeling stability. To tackle these issues, we propose an SMPL-prior-driven implicit neural representation framework. Our method introduces signed distance regularization and bounding-box-guided rendering to enforce structural consistency of human bodies; employs ray-consistency regularization to strengthen multi-view geometric coherence; and proposes a lighting-robust saturation regularization to improve appearance disentanglement. Extensive experiments on both real-world and synthetic datasets demonstrate state-of-the-art performance—achieving significant gains in reconstruction accuracy and novel-view synthesis quality—while maintaining strong robustness under extremely sparse input conditions.

Technology Category

• Computer Vision: Multi-modal Vision
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Machine Learning: Learning with Manifolds

Application Category

• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

We present a method for recovering the shape and radiance of a scene consisting of multiple people given solely a few images. Multi-human scenes are complex due to additional occlusion and clutter. For single-human settings, existing approaches using implicit neural representations have achieved impressive results that deliver accurate geometry and appearance. However, it remains challenging to extend these methods for estimating multiple humans from sparse views. We propose a neural implicit reconstruction method that addresses the inherent challenges of this task through the following contributions: First, we propose to use geometry constraints by exploiting pre-computed meshes using a human body model (SMPL). Specifically, we regularize the signed distances using the SMPL mesh and leverage bounding boxes for improved rendering. Second, we propose a ray regularization scheme to minimize rendering inconsistencies, and a saturation regularization for robust optimization in variable illumination. Extensive experiments on both real and synthetic datasets demonstrate the benefits of our approach and show state-of-the-art performance against existing neural reconstruction methods.

Problem

• Recover scene shape and radiance
• Handle multi-human scene complexity
• Extend neural methods to sparse views

Innovation

• Uses SMPL for geometry constraints
• Implements ray regularization scheme
• Applies saturation regularization technique