Hand-Aware Transition Modeling for Bimanual Procedural Anomaly Detection

2026-09-17

AI Summary

为解决双手装配过程中的异常检测问题，提出HACT模型，通过预测每只手的动作事件并计算其语义和时间上的意外程度来识别异常。

Abstract

Procedural anomaly detection in bimanual assembly requires judging each hand action against the execution so far. A corrective action may look unusual in isolation, while a visually plausible action can violate the order of the procedure. We present HACT, a transition model over predicted per-hand events. A role-preserving history keeps the concurrent responsibilities of both hands, and a marked temporal point process assigns each observed transition a semantic and temporal surprisal. A supervised evidence head and a two-state filter convert these surprisals into per-hand anomaly posteriors. A recovery-aware protocol on predicted events and participant-disjoint folds reports the recovery false-positive rate at an operating point selected on validation participants. On two bimanual power-tool procedures HACT has the highest AUPRC and F1 among the compared methods and the fewest recovery alarms. Applied without retraining to a different assembly order of the same product, it retains the highest AUPRC and F1. The source code is available at https://github.com/Kratos-Wen/HACT.

Problem

• bimanual assembly
• procedural anomaly detection
• hand action

Innovation

• HACT
• bimanual procedural anomaly detection
• role-preserving history
• marked temporal point process
• recovery-aware protocol