PointFix: Learning to Fix Domain Bias for Robust Online Stereo Adaptation

2022-07-27

European Conference on Computer Vision

AI Summary

To address domain shift between synthetic training and real-world test data in online stereo matching—particularly severe in dynamic object regions, leading to poor adaptability and degraded accuracy—this paper proposes a meta-learning-driven online adaptation framework. Our method introduces three key innovations: (1) a plug-and-play point selection auxiliary network for robust initialization; (2) the first gradient-driven, local-bias-aware refinement mechanism, enabling model-agnostic online optimization; and (3) differentiable point selection coupled with local gradient backpropagation for efficient adaptation. Evaluated under short-, medium-, and long-term online sequence settings, our approach achieves state-of-the-art performance across all benchmarks. It significantly improves matching accuracy in dynamic regions and enhances convergence robustness, demonstrating superior generalization and real-time adaptability without architectural modifications to the base stereo matcher.

Abstract

Online stereo adaptation tackles the domain shift problem, caused by different environments between synthetic (training) and real (test) datasets, to promptly adapt stereo models in dynamic real-world applications such as autonomous driving. However, previous methods often fail to counteract particular regions related to dynamic objects with more severe environmental changes. To mitigate this issue, we propose to incorporate an auxiliary point-selective network into a meta-learning framework, called PointFix, to provide a robust initialization of stereo models for online stereo adaptation. In a nutshell, our auxiliary network learns to fix local variants intensively by effectively back-propagating local information through the meta-gradient for the robust initialization of the baseline model. This network is model-agnostic, so can be used in any kind of architectures in a plug-and-play manner. We conduct extensive experiments to verify the effectiveness of our method under three adaptation settings such as short-, mid-, and long-term sequences. Experimental results show that the proper initialization of the base stereo model by the auxiliary network enables our learning paradigm to achieve state-of-the-art performance at inference.

Problem

• Addresses domain shift in online stereo adaptation
• Fixes dynamic object regions with severe environmental changes
• Provides robust initialization for stereo models

Innovation

• Auxiliary point-selective network for robust initialization
• Meta-learning framework with meta-gradient backpropagation
• Model-agnostic plug-and-play architecture compatibility