SelfHVD: Self-Supervised Handheld Video Deblurring for Mobile Phones

2025-08-11

AI Summary

Existing self-supervised video deblurring methods for handheld videos suffer from limited generalization due to domain gaps between synthetically generated and real-world motion blur. Method: We propose an unpaired self-supervised framework comprising three key components: (1) a self-augmentation strategy leveraging temporal consistency to generate pseudo-sharp frames and inter-frame misalignment labels; (2) a self-constrained spatial consistency regularization that explicitly enforces pixel-level motion continuity; and (3) a cross-domain hybrid dataset combining synthetic and real blurred videos to bridge the blur domain gap. Contribution/Results: Evaluated on multiple real handheld video benchmarks, our method significantly outperforms state-of-the-art self-supervised approaches, effectively mitigating domain shift and achieving more robust and photorealistic dynamic deblurring.

Technology Category

• Computer Vision: Motion & Tracking
• Machine Learning: Unsupervised & Self-Supervised Learning
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Vertical and domain-specific search
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

Shooting video with a handheld mobile phone, the most common photographic device, often results in blurry frames due to shaking hands and other instability factors. Although previous video deblurring methods have achieved impressive progress, they still struggle to perform satisfactorily on real-world handheld video due to the blur domain gap between training and testing data. To address the issue, we propose a self-supervised method for handheld video deblurring, which is driven by sharp clues in the video. First, to train the deblurring model, we extract the sharp clues from the video and take them as misalignment labels of neighboring blurry frames. Second, to improve the model's ability, we propose a novel Self-Enhanced Video Deblurring (SEVD) method to create higher-quality paired video data. Third, we propose a Self-Constrained Spatial Consistency Maintenance (SCSCM) method to regularize the model, preventing position shifts between the output and input frames. Moreover, we construct a synthetic and a real-world handheld video dataset for handheld video deblurring. Extensive experiments on these two and other common real-world datasets demonstrate that our method significantly outperforms existing self-supervised ones. The code and datasets are publicly available at https://github.com/cshonglei/SelfHVD.

Problem

• Addresses blur in handheld mobile videos from shaking
• Bridges blur domain gap in training and testing data
• Enhances deblurring via self-supervised sharp clues extraction

Innovation

• Self-supervised sharp clues extraction for deblurring
• Self-Enhanced Video Deblurring for better training data
• Self-Constrained Spatial Consistency to prevent shifts