Computational modeling of early language learning from acoustic speech and audiovisual input without linguistic priors

2026-03-09

AI Summary

This study investigates infant-like early language acquisition from raw audio and audiovisual inputs without any linguistic priors. To this end, we propose a self-supervised, visually grounded multimodal computational model that relies solely on shared learning principles, eschewing predefined linguistic structures to learn language perception directly from real-world audiovisual signals. The model successfully replicates multiple empirical phenomena observed in infant language development, thereby demonstrating the feasibility of language acquisition in the absence of explicit linguistic priors. Our approach significantly enhances the cognitive plausibility of computational models and improves their alignment with human language acquisition mechanisms.

Technology Category

• Natural Language Processing: Language Grounding & Multi-modal NLP
• Computer Vision: Language and Vision
• Machine Learning: Multimodal Learning

Application Category

• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Learning to understand speech appears almost effortless for typically developing infants, yet from an information-processing perspective, acquiring a language from acoustic speech is an enormous challenge. This chapter reviews recent developments in using computational models to understand early language acquisition from speech and audiovisual input. The focus is on self-supervised and visually grounded models of perceptual learning. We show how these models are becoming increasingly powerful in learning various aspects of speech without strong linguistic priors, and how many features of early language development can be explained through a shared set of learning principles-principles broadly compatible with multiple theories of language acquisition and human cognition. We also discuss how modern learning simulations are gradually becoming more realistic, both in terms of input data and in linking model behavior to empirical findings on infant language development.

Problem

• language acquisition
• computational modeling
• speech perception
• audiovisual input
• self-supervised learning

Innovation

• self-supervised learning
• visually grounded models
• language acquisition
• perceptual learning
• computational modeling