TimeMaster: Training Time-Series Multimodal LLMs to Reason via Reinforcement Learning

2025-06-16

AI Summary

Existing multimodal large language models (MLLMs) for time series suffer from limited reasoning capabilities under dynamic temporal patterns, semantic ambiguity, and insufficient temporal priors. Method: We propose a reinforcement learning–driven structured reasoning framework featuring a novel three-stage output format—chain-of-reasoning, classification decision, and domain-knowledge expansion—alongside a composite reward function and Group Relative Policy Optimization (GRPO) for fine-grained, stable time-series reasoning enhancement. The model is initialized via supervised fine-tuning (SFT) and further trained with GRPO on Qwen2.5-VL-3B-Instruct. Contribution/Results: Our approach achieves state-of-the-art performance on TimerBed, a benchmark comprising six real-world time-series classification tasks, outperforming classical time-series models by 14.6% and few-shot GPT-4o by 7.3%. It additionally delivers interpretable reasoning traces, context-aware explanations, and domain-aligned knowledge integration.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning
• Natural Language Processing: Language Grounding & Multi-modal NLP

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Time-series reasoning remains a significant challenge in multimodal large language models (MLLMs) due to the dynamic temporal patterns, ambiguous semantics, and lack of temporal priors. In this work, we introduce TimeMaster, a reinforcement learning (RL)-based method that enables time-series MLLMs to perform structured, interpretable reasoning directly over visualized time-series inputs and task prompts. TimeMaster adopts a three-part structured output format, reasoning, classification, and domain-specific extension, and is optimized via a composite reward function that aligns format adherence, prediction accuracy, and open-ended insight quality. The model is trained using a two-stage pipeline: we first apply supervised fine-tuning (SFT) to establish a good initialization, followed by Group Relative Policy Optimization (GRPO) at the token level to enable stable and targeted reward-driven improvement in time-series reasoning. We evaluate TimeMaster on the TimerBed benchmark across six real-world classification tasks based on Qwen2.5-VL-3B-Instruct. TimeMaster achieves state-of-the-art performance, outperforming both classical time-series models and few-shot GPT-4o by over 14.6% and 7.3% performance gain, respectively. Notably, TimeMaster goes beyond time-series classification: it also exhibits expert-like reasoning behavior, generates context-aware explanations, and delivers domain-aligned insights. Our results highlight that reward-driven RL can be a scalable and promising path toward integrating temporal understanding into time-series MLLMs.

Problem

• Addresses time-series reasoning challenges in multimodal LLMs
• Enhances structured, interpretable reasoning via reinforcement learning
• Improves performance in time-series classification and insight generation

Innovation

• Reinforcement learning optimizes time-series MLLMs
• Structured output format enhances interpretable reasoning
• Two-stage training with SFT and GRPO