MarineEVT: Advancing Event-Centric Marine Video Understanding via Visual Tool Reasoning

2026-07-27

AI Summary

This work addresses the challenges of marine video understanding—namely, difficulties in temporal modeling, scarcity of annotations, strong reliance on domain knowledge, and sparsity of critical events—which hinder existing vision-language models from accurately localizing and interpreting key information. To tackle these issues, the study introduces MarineEVT, the first event-centric dataset for marine video understanding, along with EVT-R1, an event-centric reasoning framework that leverages visual tools to align model behavior with human intent for precise event localization and semantic interpretation. Trained on MarineEVT via multi-task video question answering, EVT-R1 integrates temporal awareness and domain-specific knowledge, outperforming current state-of-the-art open-source and commercial models by 5.22 and 11.09 percentage points in accuracy, respectively, thereby advancing marine ecological analysis and sustainable video understanding.

Abstract

Recent Vision-Language Models (VLMs) have achieved remarkable success in visual understanding, driven by the growing availability of high-quality image-text pairs. However, the performance of VLMs often degrades in the video domain due to the essential need for temporal understanding and the scarcity of large-scale annotated video data. In this work, we focus on marine video understanding, which brings further challenges: first, it requires substantial domain expertise; and video VLMs usually struggle with localizing and interpreting critical information from marine videos, as the informative events are typically sparse, unpredictable, and unevenly distributed. To address these challenges, we carefully curate the first event-centric marine video understanding dataset called MarineEVT, which features 20K multi-task, video-level visual question-answering pairs spanning multiple dimensions of marine understanding and analysis. Meanwhile, based on MarineEVT, we decompose marine video understanding as an Event-centric Visual Tool-integrated Reasoning process EVT-R1 for short, where we leverage powerful visual tools to drive the model to localize and interpret critical information aligned with visual questions and human intent. To demonstrate its effectiveness, we compare EVT-R1 against 11 SOTA VLMs in different settings. EVT-R1 outperforms the top open-source and top commercial models by 5.22 and 11.09, respectively. MarineEVT and EVT-R1 lay the foundation for ecological discovery and marine education, fostering the development of VLMs capable of interpreting marine dynamics, reasoning about ecological interactions, and supporting sustainable ocean video understanding and analysis.

Problem

• marine video understanding
• event-centric reasoning
• visual-language models
• temporal understanding
• sparse events

Innovation

• event-centric reasoning
• visual tool integration
• marine video understanding
• video-language models
• temporal localization