EgoPressDiff: Multimodal Video Diffusion for Egocentric UV-Domain Hand-Pressure Estimation

2026-06-04

AI Summary

This work addresses the limitations of existing methods for hand–object contact pressure estimation from first-person views, which suffer from quantization errors and temporal inconsistency due to discretization and frame-wise processing. To overcome these issues, the authors propose a conditional video diffusion framework that integrates hand pose, 3D mesh vertices, and depth information to generate continuous, temporally coherent, and physically plausible pressure map sequences in the UV domain. A multimodal conditioning strategy combined with distribution-calibrated spatial layers effectively aligns and fuses heterogeneous features. Evaluated on the EgoPressure dataset, the method achieves a relative improvement of over 34% in volumetric IoU and significantly reduces mean absolute error while maintaining high temporal consistency.

Abstract

Estimating hand-surface contact pressure from an egocentric view is crucial for AR/VR devices, robotic imitation, and ergonomic analysis. Existing methods often discretize pressure signal and process frames independently, leading to quantization errors and temporal inconsistencies. We present \emph{EgoPressDiff}, a conditional video diffusion framework that generates UV-pressure maps from visual input. The core of our approach is a multi-modal conditioning strategy, introducing a PoseNet and a Vertex Encoder to efficiently extract features from hand pose and 3D mesh vertices. These signals, along with depth information, guide the generative process to ensure the pressure fields are physically grounded. To effectively fuse these heterogeneous features, we further propose a Distribution-Calibrated Spatial Layer, which aligns their statistical properties before combination. Evaluated on the EgoPressure ego-view setting, EgoPressDiff achieves state-of-the-art results, improving Volumetric IoU by over 34\% relative to prior baseline, while reducing MAE and maintaining high temporal accuracy. Our project page is at https://egopressdiff.github.io/.

Problem

• egocentric vision
• hand-pressure estimation
• UV-domain
• temporal consistency
• contact pressure

Innovation

• video diffusion
• multimodal conditioning
• UV-pressure estimation
• egocentric vision
• distribution-calibrated fusion