TAMMs: Temporal-Aware Multimodal Model for Satellite Image Change Understanding and Forecasting

2025-06-23

AI Summary

Satellite image time-series analysis demands fine-grained spatiotemporal reasoning—yet current multimodal large language models (MLLMs) exhibit limited capability in this domain. To address this, we propose TAMMs, the first framework jointly modeling change understanding and future scene generation from satellite time series. Our method introduces: (1) a lightweight temporal module that enhances a frozen MLLM via structured sequence encoding and context-aware prompting; and (2) a semantic-fusion control injection mechanism within an enhanced ControlNet architecture, implementing dual-path conditional generation that synergistically integrates high-level semantics and structural priors to improve both temporal consistency and semantic fidelity. Extensive experiments across multiple satellite time-series benchmarks demonstrate that TAMMs significantly outperforms state-of-the-art MLLMs in both change detection and future image prediction, validating its effectiveness and advancement in modeling complex dynamic spatiotemporal processes.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Computer Vision: Multi-modal Vision
• Planning, Routing, and Scheduling: Planning with Language Models

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information

Abstract

Satellite image time-series analysis demands fine-grained spatial-temporal reasoning, which remains a challenge for existing multimodal large language models (MLLMs). In this work, we study the capabilities of MLLMs on a novel task that jointly targets temporal change understanding and future scene generation, aiming to assess their potential for modeling complex multimodal dynamics over time. We propose TAMMs, a Temporal-Aware Multimodal Model for satellite image change understanding and forecasting, which enhances frozen MLLMs with lightweight temporal modules for structured sequence encoding and contextual prompting. To guide future image generation, TAMMs introduces a Semantic-Fused Control Injection (SFCI) mechanism that adaptively combines high-level semantic reasoning and structural priors within an enhanced ControlNet. This dual-path conditioning enables temporally consistent and semantically grounded image synthesis. Experiments demonstrate that TAMMs outperforms strong MLLM baselines in both temporal change understanding and future image forecasting tasks, highlighting how carefully designed temporal reasoning and semantic fusion can unlock the full potential of MLLMs for spatio-temporal understanding.

Problem

• Enhancing MLLMs for satellite image temporal change understanding
• Jointly addressing change understanding and future scene generation
• Improving spatio-temporal reasoning with lightweight temporal modules

Innovation

• Lightweight temporal modules enhance frozen MLLMs
• Semantic-Fused Control Injection for adaptive generation
• Dual-path conditioning ensures consistent image synthesis