Identity-Centric Video Summarization via Hierarchical Fusion of Biometric, Appearance, and 3D Body Features

2026-09-22

AI Summary

该研究通过融合生物识别、外观和3D身体特征，提出了一种基于多目标跟踪和个人再识别的视频摘要算法，以解决视频中身份识别与跟踪的问题。

Abstract

This work presents a video summarization algorithm based on multi-object tracking and person reidentification. We integrate facial embeddings, 3D body-shape features, and visual appearance into a unified tracking framework. These representations enable hierarchical identity assignment and tracking through bidirectional anchoring, which robustly recovers trajectories under severe occlusion or low visual quality. From these stable trajectories, we generate a compact set of summaries for each identity. We select keyframes using a multi-factor weighting scheme that optimizes biometric clarity, social interaction, and motion dynamics, while Adaptive Non-Maximum Suppression ensures temporal diversity. Evaluation on a custom dataset demonstrates tracking stability, achieving an IDF1 of 97.89% and a MOTA of 95.79%. Compared to Top-K selection, our algorithm also increases visual diversity by 146%, temporal coverage by 89%, and information retrievability by 3.5%.

Problem

• video summarization
• identity-centric
• multi-object tracking
• person reidentification
• trajectory recovery

Innovation

• multi-object tracking
• person reidentification
• hierarchical identity assignment
• bidirectional anchoring
• Adaptive Non-Maximum Suppression