Design Choices That Matter: A Functional ANOVA Analysis for Remote Sensing Multi-Label Classification

2026-08-05

AI Summary

This study addresses the limited generalizability of benchmark rankings and the lack of systematic understanding of design choices in deep learning models for multi-label remote sensing image classification. It introduces functional Analysis of Variance (fANOVA) to this domain for the first time, quantifying the individual and interactive effects of key design factors—including network architecture, fine-tuning strategy, learning strategy, and initialization—on model performance. Leveraging seven datasets, the authors construct meta-representations and apply hierarchical clustering to uncover relationships between these factors and intrinsic dataset properties such as spatial scale, resolution, and label complexity. The analysis reveals that fine-tuning and architecture dominate performance on large-scale data, initialization is most critical under data scarcity, and medium-sized datasets benefit from synergistic interactions between architecture and learning strategy. Furthermore, datasets naturally cluster according to their response patterns, offering interpretable guidance for model design in remote sensing multi-label tasks.

Abstract

Benchmarking deep learning (DL) models for multi-label classification (MLC) of remote sensing images (RSI) typically yields rankings that do not generalize beyond the evaluated datasets. In this work, we move beyond rankings by employing functional analysis of variance (fANOVA) to systematically quantify the contributions of individual design choices and their interactions to performance variability. We conduct two empirical analyses covering 48 and 20 DL models, respectively, spanning design choices such as network architecture, fine-tuning strategy, learning strategy, and initialization. By applying fANOVA across seven MLC RSI datasets, we construct dataset meta-representations that capture design-choice sensitivity profiles. Hierarchical clustering of these meta-representations reveals that datasets naturally group according to how they respond to design decisions, with patterns strongly linked to intrinsic dataset properties such as scale, spatial resolution, and label space complexity. Our findings show that for large-scale datasets, fine-tuning strategy and architecture are dominant factors, while in data-limited regimes, initialization becomes decisive. For intermediate regimes, the interaction between architecture and learning strategy governs performance.

Problem

• multi-label classification
• remote sensing images
• deep learning
• benchmarking
• design choices

Innovation

• functional ANOVA
• multi-label classification
• remote sensing images
• design choice analysis
• dataset meta-representation