Gradient-Guided Decoupled Adaptation for Geospatial Vision-Language Models

2026-09-26

AI Summary

This study addresses the optimization difficulties and task interference caused by gradient heterogeneity in multi-task adaptation of geospatial vision-language models. To this end, we propose G2DA, a framework that introduces a novel gradient-based cross-modal task grouping mechanism to decouple tasks. By integrating similarity-aware curriculum learning with a bidirectional rehearsal strategy, G2DA effectively mitigates recency effects during sequential optimization and alleviates catastrophic forgetting. The proposed method seamlessly adapts to mainstream architectures such as InternVL3. Extensive evaluations across 24 experimental configurations on three benchmarks demonstrate that G2DA consistently outperforms existing approaches, surpassing the strongest baseline by 3.08%, 4.30%, and 2.81 percentage points, respectively. These results validate both the efficiency and generalization capability of the framework for geospatial multi-task learning.

Abstract

Existing geospatial vision-language models (Geo-VLMs) typically optimize diverse geospatial tasks through a unified multi-task adaptation paradigm without explicitly accounting for the heterogeneous optimization characteristics. Our empirical observations reveal heterogeneous gradient characteristics across tasks, including vision-language differences, intra-branch gradient relationships, and task interference, which hinder effective multi-task optimization. Motivated by these observations, we propose Gradient-Guided Decoupled Adaptation (G2DA), a gradient-aware optimization framework for multi-task Geo-VLM learning. G2DA first partitions tasks into vision- and language-centric groups through gradient-guided cross-modal decoupling. It then constructs modality-specific curricula based on task gradient similarity and employs bidirectional rehearsal to mitigate the recency effects introduced by sequential optimization. We evaluate G2DA on three Geo-VLM benchmarks using six InternVL3 and Qwen3.5-VL variants, along with GeoChat and GeoLLaVA. Across all 24 benchmark-model combinations, G2DA consistently outperforms representative baselines, improving over the strongest competitor by 3.08, 4.30, and 2.81 percentage points on UrBench-MCQ, XLRS-Bench-Lite, and VRS-Bench-VQA, respectively. These results demonstrate the effectiveness of gradient-guided task organization for Geo-VLM adaptation.

Problem

• Geospatial Vision-Language Models
• Multi-task Optimization
• Gradient Heterogeneity
• Task Interference

Innovation

• Gradient-Guided Decoupled Adaptation
• Geospatial Vision-Language Models
• Cross-modal Decoupling
• Curriculum Learning
• Bidirectional Rehearsal