HOPE: Hand-Object Pressure Estimation from Monocular Videos

2026-08-06

AI Summary

This work proposes the first method to estimate physical hand–object interaction pressure from monocular video, overcoming limitations of prior approaches that are restricted to planar surfaces or single-frame inputs. The task is formulated as a hand-centric temporal prediction problem, yielding contact state and object-agnostic normal pressure directly on hand mesh vertices. Key innovations include the first alignment of multi-source tactile and contact annotations to hand vertex space, a vertex-anchored video Transformer architecture, and a contact-gating mechanism that enforces zero pressure in non-contact regions. Evaluated on benchmarks such as OpenTouch and PressureVisionDB, the method significantly outperforms baselines that only predict contact or planar pressure, generalizes effectively to ungloved first-person and in-the-wild videos, and accurately recovers both contact and pressure distributions.

Abstract

Estimating physical pressure from vision is essential for understanding contact-rich hand-object interaction. However, prior vision-based pressure estimation methods are largely limited to planar surfaces and single image input, making them difficult to apply to dynamic hand-object interaction with diverse objects. We instead formulate pressure estimation as a hand-centric video prediction problem with monocular video as input. This formulation predicts temporally evolving per-vertex normal pressure and contact directly on the hand mesh, yielding a unified output space independent of object shape and sensor layout. Building on this formulation, we propose \textbf{HOPE}, a framework with two key components. First, we lift tactile-glove pressure, planar-sensor pressure, and distance-based hand-object contact annotations into a shared hand vertex space, allowing bare-hand contact data to regularize pressure learning where metric labels are unavailable. Second, we introduce a vertex-anchored video transformer that treats each vertex as a persistent token, aggregates visual features and hand pose over time, and uses a contact-gated pressure head to enforce that pressure vanishes without contact. Experiments on OpenTouch, PressureVisionDB, and hand-object contact benchmarks validate HOPE across object-pressure, surface-pressure, and contact-supervised HOI settings. Despite using metric pressure supervision primarily from gloved-hand videos, HOPE generalizes to bare-hand egocentric and in-the-wild videos, producing joint contact and pressure predictions beyond the scope of contact-only or planar-pressure baselines.

Problem

• pressure estimation
• hand-object interaction
• monocular video
• contact-rich interaction
• tactile sensing

Innovation

• pressure estimation
• hand-object interaction
• monocular video
• vertex-anchored transformer
• contact-aware prediction