🤖 AI Summary
In few-shot learning, deep networks suffer from channel bias—over-reliance on source-task-discriminative channels—leading to feature redundancy: most channels exhibit high intra-class variance and low inter-class separability, thereby hindering adaptation to novel tasks. Method: We establish, for the first time, the causal chain “channel bias → feature redundancy → performance degradation,” theoretically proving that redundancy is exacerbated under low-data regimes. We reveal the “less-is-more” principle: retaining only 1–5% of the most discriminative channels significantly improves accuracy, while >95% contribute negatively—a detrimental effect that attenuates with increasing sample size. Accordingly, we propose Augmented Feature Importance Adjustment (AFIA), a theoretically grounded method integrating data augmentation and soft channel masking to suppress redundancy. Results: AFIA achieves consistent and significant improvements across standard few-shot benchmarks, offering both a new theoretical principle and a practical tool for few-shot learning.
📝 Abstract
Deep neural networks often fail to adapt representations to novel tasks under distribution shifts, especially when only a few examples are available. This paper identifies a core obstacle behind this failure: channel bias, where networks develop a rigid emphasis on feature dimensions that were discriminative for the source task, but this emphasis is misaligned and fails to adapt to the distinct needs of a novel task. This bias leads to a striking and detrimental consequence: feature redundancy. We demonstrate that for few-shot tasks, classification accuracy is significantly improved by using as few as 1-5% of the most discriminative feature dimensions, revealing that the vast majority are actively harmful. Our theoretical analysis confirms that this redundancy originates from confounding feature dimensions-those with high intra-class variance but low inter-class separability-which are especially problematic in low-data regimes. This "less is more" phenomenon is a defining characteristic of the few-shot setting, diminishing as more samples become available. To address this, we propose a simple yet effective soft-masking method, Augmented Feature Importance Adjustment (AFIA), which estimates feature importance from augmented data to mitigate the issue. By establishing the cohesive link from channel bias to its consequence of extreme feature redundancy, this work provides a foundational principle for few-shot representation transfer and a practical method for developing more robust few-shot learning algorithms.
