🤖 AI Summary
This work addresses the open question of whether, under a fixed parameter budget, model capacity should be concentrated in a single wide path or distributed across multiple narrow branches. The authors propose the Multi-Narrow transformation, which restructures standard CNNs into ensembles of independent narrow pathways, and conduct a systematic evaluation comparing single-wide and multi-narrow architectures under comparable total parameter counts across varying data regimes, network designs, and datasets. Experimental results demonstrate that Multi-Narrow significantly outperforms baseline single-wide models in low-data settings, owing to reduced inter-path feature redundancy and enhanced diversity, which collectively improve generalization. However, it underperforms when ample training data is available. The study delineates the effective operating regime of multi-narrow structures and elucidates their underlying mechanisms.
📝 Abstract
Single-model ensembles (SMEs) have attracted attention as a way to approximate some of the benefits of deep ensembles within a single network. However, under an approximately matched parameter budget, it remains unclear whether model capacity should be concentrated in a single wide pathway or redistributed into many narrow and independent members. We investigate this question through the Multi-Narrow (MN) transformation, which converts a baseline CNN into an SME of narrow, path-wise independent branches while approximately preserving the dominant parameter budget. We systematically compare Single-Wide and Multi-Narrow configurations across different training-data regimes, architectures, and datasets. The results show that the effectiveness of MN is strongly data-dependent: weakly partitioned or baseline-wide models are preferable in data-rich settings, whereas highly partitioned MN models consistently outperform the baseline in low-data settings. This tendency is reproduced across multiple CNN architectures and image-classification datasets, suggesting that it is not specific to a single benchmark or model family. Analysis of internal representations shows that high-MN models learn more diverse and less redundant path-wise features. In low-data regimes, this diversity is broadly utilized and improves generalization, whereas in data-rich regimes, training becomes imbalanced and prediction is dominated by a small subset of paths. These findings clarify when and why Multi-Narrow transformation is effective, and provide practical guidance for allocating model capacity between width and member multiplicity under a limited budget.