RDVSv2: A Large-scale Benchmark for RGB-D Video Salient Object Detection

2026-07-28

AI Summary

Existing RGB-D video salient object detection datasets suffer from limited scale, low annotation quality, and geometrically inconsistent depth maps, hindering robust modeling in complex scenes. To address these limitations, this work introduces RDVSv2, a large-scale benchmark comprising 249 videos with 29,077 densely annotated frames. The dataset leverages stereoscopic video to generate geometrically consistent depth maps and employs eye-tracking guidance to produce high-quality saliency masks. Furthermore, the paper proposes a strong baseline method built upon SAM2, which integrates RGB, depth, and optical flow modalities through parameter-efficient fine-tuning. RDVSv2 substantially elevates task difficulty and realism, and the proposed approach achieves state-of-the-art performance on both RDVSv2 and existing benchmarks, offering a high-quality resource and a powerful foundation for multimodal video understanding.

Abstract

We introduce RDVSv2, a large-scale benchmark for RGB-D video salient object detection (RGB-D VSOD) with dense frame-level annotations. Existing datasets in this emerging field are often limited in scale and annotation quality, while also relying on less geometry-consistent depth cues. To address these limitations, RDVSv2 is built from publicly accessible stereoscopic online videos and contains 249 video sequences with 29,077 annotated frames. It includes depth maps derived from stereoscopic videos, together with frame-wise salient object masks annotated with eye-tracking guidance. Compared with existing datasets, RDVSv2 is much larger in scale and covers more diverse and challenging scenarios. In addition, we establish a strong baseline for RGB-D VSOD based on Segment Anything Model 2 (SAM2). Specifically, we employ a parameter-efficient fine-tuning (PEFT) strategy to adapt the SAM2 encoder to jointly encode RGB, depth, and optical flow cues. Extensive experiments show that RDVSv2 is substantially more challenging for existing RGB-D VSOD methods. Meanwhile, the proposed baseline achieves state-of-the-art results on RDVSv2 and existing RGB-D VSOD benchmarks. We hope that RDVSv2 and the provided baseline will serve as useful resources for future research on RGB-D VSOD and related multi-modal video understanding tasks. Our dataset and code will be available at https://github.com/ltynick/RDVSv2.

Problem

• RGB-D video salient object detection
• large-scale benchmark
• dense frame-level annotations
• depth maps
• stereoscopic videos

Innovation

• RGB-D video salient object detection
• large-scale benchmark
• stereoscopic depth
• parameter-efficient fine-tuning
• SAM2