Incorporating brain-inspired mechanisms for multimodal learning in artificial intelligence
Current multimodal fusion methods in AI predominantly employ static weighting strategies, overlooking the neuroscientific principle of inverse effectiveness—the phenomenon wherein weaker unimodal cues elicit stronger multimodal integration gains. To address this, we propose Inverse-Effectiveness-Driven Multimodal Fusion (IEMF), the first approach to formalize this principle as a learnable, differentiable dynamic fusion mechanism that adaptively modulates integration strength based on unimodal confidence estimates. The IEMF module integrates attention mechanisms, dynamic gating, and cross-modal confidence estimation, and is compatible with both artificial neural networks (ANNs) and spiking neural networks (SNNs). Evaluated on audio-visual classification, continual learning, and multimodal question answering, IEMF significantly enhances model robustness—particularly under noisy or sparse inputs—while reducing computational overhead by up to 50%. Moreover, it demonstrates strong generalization across diverse tasks and modalities.