Learning Optical Flow Field via Neural Ordinary Differential Equation

2025-06-03

AI Summary

Existing optical flow methods employ fixed-step iterative refinement, which hinders adaptability to input complexity and limits both modeling accuracy and computational efficiency. This work introduces Neural Ordinary Differential Equations (Neural ODEs) to optical flow estimation for the first time, formulating flow field optimization as a continuous dynamical evolution process. By leveraging adaptive ODE solvers (e.g., DOPRI5), our approach enables data-driven, variable computation depth while preserving equivalence to RNN-based architectures—yet with substantially enhanced modeling flexibility. Evaluated on standard benchmarks including Sintel and KITTI, our method consistently outperforms baseline and state-of-the-art models: it achieves higher accuracy with a single forward evolution and reduces inference FLOPs. The core contribution is the first Neural ODE-based optical flow framework, enabling joint improvement in accuracy and efficiency through adaptive, continuous-time flow optimization.

Technology Category

• Computer Vision: Learning & Optimization for CV
• Search and Optimization: Learning to Search
• Machine Learning: Deep Neural Architectures and Foundation Models

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation

Abstract

Recent works on optical flow estimation use neural networks to predict the flow field that maps positions of one image to positions of the other. These networks consist of a feature extractor, a correlation volume, and finally several refinement steps. These refinement steps mimic the iterative refinements performed by classical optimization algorithms and are usually implemented by neural layers (e.g., GRU) which are recurrently executed for a fixed and pre-determined number of steps. However, relying on a fixed number of steps may result in suboptimal performance because it is not tailored to the input data. In this paper, we introduce a novel approach for predicting the derivative of the flow using a continuous model, namely neural ordinary differential equations (ODE). One key advantage of this approach is its capacity to model an equilibrium process, dynamically adjusting the number of compute steps based on the data at hand. By following a particular neural architecture, ODE solver, and associated hyperparameters, our proposed model can replicate the exact same updates as recurrent cells used in existing works, offering greater generality. Through extensive experimental analysis on optical flow benchmarks, we demonstrate that our approach achieves an impressive improvement over baseline and existing models, all while requiring only a single refinement step.

Problem

• Predicting optical flow field dynamically using neural ODE
• Overcoming fixed-step refinement limitations in flow estimation
• Achieving better performance with fewer refinement steps

Innovation

• Uses neural ODE for flow derivative prediction
• Dynamically adjusts computation steps per data
• Replicates updates with single refinement step