Reuse or Relearn? A Spectral View of Earth Observation Foundation Models

2026-09-26

AI Summary

This study addresses the limitation that downstream accuracy alone cannot reveal how pretrained representations are reused or relearned during the fine-tuning of Earth Observation (EO) foundation models. To this end, it proposes a spectral diagnostic evaluation framework that leverages singular value decomposition to quantify subspace retention and weight update distributions before and after model adaptation, benchmarking against general-purpose vision models such as CLIP and DINO. The findings demonstrate that fine-tuning EO models relies on substantial high-rank updates with low structural retention. Furthermore, preserving pretrained subspaces is shown to enhance parameter-efficient fine-tuning performance. Consequently, this work advocates incorporating representation reusability into standard evaluation protocols for EO foundation models.

Abstract

Foundation models are rarely used as generic, frozen feature extractors; instead, they are fine-tuned for the target downstream application. This practice is particularly prevalent in Earth observation (EO), and it raises a question that downstream accuracy alone cannot answer: does fine-tuning reuse the pretrained representation, or does it relearn a new one? We study this with spectral diagnostics that compare a model before and after adaptation, quantifying how well its dominant singular subspaces are preserved, how broadly the weight update is distributed, and how large it is. Using natural image models such as CLIP and DINO as a reference, we find that, under the evaluated fine-tuning settings, EO models undergo far larger, higher-rank updates and retain much less of their pretrained structure, so their downstream performance is often obtained with substantial changes to the pretrained weight structure. The diagnostics further provide insight into how cheaply a model can be adapted: where the pretrained subspaces are preserved, adapting a small fraction of the parameters can match full fine-tuning, and where they are not, it can fall behind. More broadly, foundation models, and EO foundation models in particular, should be assessed not only by benchmark accuracy, but also by how reusable their pretrained representation is.

Problem

• Earth observation foundation models
• fine-tuning
• representation reuse
• spectral diagnostics

Innovation

• Spectral Diagnostics
• Earth Observation Foundation Models
• Fine-tuning Dynamics
• Representation Reusability
• Parameter-Efficient Adaptation