DINOde: Continuous Vision-Text Alignment for Open-Vocabulary Semantic Segmentation

2026-07-23

AI Summary

DINOv3 lacks native text-alignment capability, limiting its direct applicability to open-vocabulary semantic segmentation. To address this, this work proposes DINOde, a novel framework that introduces ordinary differential equations (ODEs) into vision–language alignment for the first time. By modeling continuous feature trajectories, DINOde avoids the manifold entanglement caused by discrete projection approaches. The method integrates two complementary components—semantic textual flow and global contextual flow—and incorporates velocity tangent-space projection to preserve the hyperspherical geometry of features. Jointly leveraging DINOv3 and CLIP, DINOde achieves state-of-the-art performance across multiple open-vocabulary semantic segmentation benchmarks, demonstrating more robust and precise cross-modal alignment.

Abstract

Open-vocabulary semantic segmentation (OVSS) leverages textual semantics to segment objects beyond predefined categories. While the self-supervised model DINOv3 provides strong structured visual representations, its lack of native textual alignment hinders its direct application to OVSS. To bridge this gap, we propose DINOde, an ODE-based framework that continuously aligns CLIP text embeddings with the DINO visual manifold. Our approach employs two complementary components: (i) Semantic Text Flow (STF), which evolves text embeddings toward the DINO manifold through a continuous ODE trajectory, and (ii) Global Context Flow (GCF), which progressively refines the holistic image representation carried by DINO's CLS token. To preserve the hyperspherical geometry of the feature space during this evolution, we further introduce Velocity Tangent Projection, which constrains the learned velocity field to the tangent space. By modeling alignment as a continuous trajectory, DINOde avoids the manifold entanglement inherent in discrete MLP projections and yields more robust cross-modal alignment. Extensive experiments demonstrate that DINOde consistently outperforms existing methods and achieves state-of-the-art performance across multiple OVSS benchmarks. The code is available at https://github.com/yoon307/DINOde.

Problem

• open-vocabulary semantic segmentation
• vision-text alignment
• DINOv3
• CLIP
• continuous alignment

Innovation

• continuous alignment
• ODE-based framework
• vision-text alignment
• open-vocabulary segmentation
• manifold geometry