How Usable Are Geospatial Foundation Models? A Systematic Evaluation of 89 Models

2026-08-04

AI Summary

This study addresses the gap between technical performance and real-world usability in geospatial foundation models (GeoFMs), as current evaluations predominantly focus on benchmark metrics while overlooking the practical needs of end users such as ecologists. Drawing on human-computer interaction theory and insights from domain expert interviews, we propose a comprehensive usability evaluation framework encompassing seven dimensions—ranging from accessibility and deployment to interactive customization, trustworthiness, and transparency—and conduct a systematic assessment of 89 GeoFMs. Our work establishes the first user-centered, multidimensional usability benchmark for GeoFMs, revealing a significant mismatch between model development and practical application: nearly one-third of models provide only source code without adequate support for real-world use. The high consistency across dimension scores demonstrates their utility as diagnostic indicators for guiding user-oriented GeoFM design.

Abstract

Geospatial foundation models (GeoFMs) offer transformative potential for environmental monitoring, yet adoption among ecologists is uneven. Most evaluations are model-centric, focusing on architecture and benchmark accuracy, which overlooks whether the systems are usable by their intended audiences. To address this gap, we first conducted a pilot expert elicitation survey with ecology and conservation scientists that helped us identify misalignments between current GeoFM development priorities and their needs. Informed by these findings and based on HCI theory, we created a seven-dimension evaluation covering Access & Deployment, Interaction & Customization, Trust & Transparency, Community & Support, Scientific Permanence, Multilingual Support, and Offline Usability. Then, two raters applied this rubric to 89 GeoFMs. We found distinct accessibility gaps where nearly a third provide no support to practitioners beyond their source code. Dimensions along which ratings were highly consistent function as field-level diagnostics, revealing where there is room for improvement for current GeoFMs.

Problem

• Geospatial foundation models
• usability
• ecology
• human-computer interaction
• model evaluation

Innovation

• Geospatial Foundation Models
• Usability Evaluation
• Human-Computer Interaction
• Ecology Applications
• Model Accessibility