OASIS: Occlusion-aware Single-image Hand Avatar Reconstruction via 3D Gaussian Splatting

2026-07-31

AI Summary

Single-image 3D hand reconstruction is an ill-posed inverse problem due to severe self-occlusion and complex non-rigid deformations, which result in sparse visual cues. This work presents the first effective integration of 3D Gaussian splatting into this task, introducing geometry-aligned visual evidence tokens, a visibility-conditioned point-image attention mechanism, and a Feature-on-Mesh representation. These components jointly enable occlusion-aware appearance reconstruction and accurate modeling of non-rigid deformations. The proposed method achieves a favorable balance between detail fidelity and computational efficiency, significantly outperforming existing approaches on challenging poses and real-world scenes. Furthermore, it supports downstream applications such as text-driven avatar generation and texture editing.

Abstract

Single-image 3D hand avatar reconstruction is fundamentally ill-posed and particularly challenging due to limited visual evidence under severe self-occlusion and the complex pose-dependent deformation of highly articulated hands. Existing methods predominantly rely on implicit NeRF-style representations, whose volumetric fitting is computationally expensive and often struggles to preserve fine-grained hand details. In this work, we present OASIS, a tailored 3D Gaussian Splatting framework for single-image hand avatar reconstruction. To faithfully encode sparse image-specific appearance cues in single-view reconstruction, we construct geometry-aligned visual evidence tokens by explicitly aligning input image observations with 3D hand geometry and context-adaptively tokenizing the resulting visual evidence. Since severe self-occlusion makes the reliability of image evidence inherently visibility-dependent, we introduce a visibility-conditioned point-image attention to reliably transfer visual evidence to geometric tokens, yielding occlusion-aware Gaussian features for faithful and robust reconstruction. To further capture non-rigid deformation of articulated hands, we introduce a Feature-on-Mesh representation to enable Gaussian deformation to be guided by local surface stretching. Under this framework, we adopt a one-shot adaptation scheme that learns a shared hand prior from multi-identity training data and then fits it to a target image for target-specific reconstruction. Extensive experiments show that OASIS outperforms existing baselines in both visual fidelity and efficiency across challenging poses and in-the-wild scenarios, and further demonstrates strong versatility in downstream applications such as text-to-avatar generation and texture editing.

Problem

• single-image hand reconstruction
• self-occlusion
• 3D hand avatar
• pose-dependent deformation
• ill-posed problem

Innovation

• 3D Gaussian Splatting
• occlusion-aware reconstruction
• single-image hand avatar
• Feature-on-Mesh
• visibility-conditioned attention