Seeing Time: Benchmarking Chronological Reasoning and Shortcut Biases in Vision-Language Models

2026-06-04

AI Summary

This work addresses the tendency of existing vision-language models to rely on superficial cues—such as image color—rather than genuine temporal logic when performing temporal reasoning. To this end, we introduce three novel fine-grained datasets encompassing historical artifacts, diverse event sequences, and rigorously aligned image-text pairs, enabling systematic evaluation of models’ temporal reasoning capabilities both within images and across modalities. Leveraging this benchmark, we uncover a pronounced reliance on non-temporal shortcuts in current models and propose a diagnostic framework that not only reveals these limitations but also guides targeted improvements. Our contribution establishes a foundation for advancing multimodal temporal logical reasoning in vision-language systems.

Abstract

Recent advancements in Vision-Language Models (VLMs) have significantly enhanced their ability to interpret complex visual semantics, yet their capacity for chronological reasoning remains under-explored. In this paper, we introduce a novel benchmark specifically designed to evaluate how VLMs perceive and reason about chronological information within and across images. Unlike existing video-based benchmarks that focus on frame sequencing, our work delves into the underlying logic of chronological judgment and the expansion toward multimodal integration. To facilitate this, we construct three specialized datasets: one containing visually similar objects spanning long historical durations, another categorized by diverse event and object types, and a third pairing images with time-sensitive news text for cross-modal alignment. Through extensive experiments, we analyze whether models exhibit performance disparities across categories and, crucially, explore whether they rely on ``incorrect shortcuts'', such as image color rather than genuine chronological features. Our results reveal that while VLMs show promise, they frequently exploit superficial cues like grayscale versus color filters to bypass authentic chronological reasoning. By providing these high-quality datasets and a rigorous evaluation framework, we offer a diagnostic tool to identify current limitations and guide the development of more robust, logically grounded multimodal models. The source code is shown in https://github.com/LuoRenqiang/ChronoVision.

Problem

• chronological reasoning
• vision-language models
• shortcut biases
• temporal understanding
• multimodal reasoning

Innovation

• chronological reasoning
• vision-language models
• shortcut bias
• multimodal benchmark
• temporal understanding