Joint Flow And Feature Refinement Using Attention For Video Restoration

2025-05-22

AI Summary

To address temporal inconsistency and flickering artifacts caused by degraded frames in video restoration, this paper proposes a multi-scale spatiotemporal modeling framework featuring bidirectional iterative refinement between optical flow and restored features. Methodologically, we introduce an occlusion-aware temporal loss to explicitly suppress flickering and design an attention-driven multi-scale feature alignment and joint optimization mechanism to reduce reliance on high-precision optical flow estimation. Our core contribution is the first-ever co-iterative refinement paradigm jointly optimizing flow and features, enabling unified modeling of inter-frame spatiotemporal consistency and fine-grained detail fidelity. Extensive experiments demonstrate state-of-the-art performance across video denoising, deblurring, and super-resolution tasks, with PSNR improvements up to 1.62 dB over prior methods. The approach significantly enhances temporal stability and perceptual quality.

Technology Category

Computer Vision: Multi-modal VisionPlanning, Routing, and Scheduling: Optimization of Spatio-temporal SystemsMachine Learning: Multimodal Learning

Application Category

Graph Algorithms and Modeling for the Web: Algorithms and analysis for heterogeneous, signed, attributed, multi-relational, temporal, higher-order, and annotated Web-related graphsSearch and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for rankingUser Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Recent advancements in video restoration have focused on recovering high-quality video frames from low-quality inputs. Compared with static images, the performance of video restoration significantly depends on efficient exploitation of temporal correlations among successive video frames. The numerous techniques make use of temporal information via flow-based strategies or recurrent architectures. However, these methods often encounter difficulties in preserving temporal consistency as they utilize degraded input video frames. To resolve this issue, we propose a novel video restoration framework named Joint Flow and Feature Refinement using Attention (JFFRA). The proposed JFFRA is based on key philosophy of iteratively enhancing data through the synergistic collaboration of flow (alignment) and restoration. By leveraging previously enhanced features to refine flow and vice versa, JFFRA enables efficient feature enhancement using temporal information. This interplay between flow and restoration is executed at multiple scales, reducing the dependence on precise flow estimation. Moreover, we incorporate an occlusion-aware temporal loss function to enhance the network's capability in eliminating flickering artifacts. Comprehensive experiments validate the versatility of JFFRA across various restoration tasks such as denoising, deblurring, and super-resolution. Our method demonstrates a remarkable performance improvement of up to 1.62 dB compared to state-of-the-art approaches.

Problem

• Enhancing degraded video frames using temporal correlations
• Improving temporal consistency in video restoration tasks
• Reducing dependence on precise flow estimation via multi-scale refinement

Innovation

• Joint flow and feature refinement using attention
• Multi-scale interplay between flow and restoration
• Occlusion-aware temporal loss for artifact reduction