Data Augmentation Using Neural Acoustic Fields With Retrieval-Augmented Pre-training

2025-04-19

AI Summary

This work addresses the performance limitation of speaker distance estimation due to scarcity of room impulse response (RIR) data. We propose a generative data augmentation method based on Neural Acoustic Fields (NAFs), featuring two key innovations: (1) the first integration of retrieval-augmented pretraining into NAF modeling, coupled with geometric conditioning to enable cross-room RIR generalization; and (2) joint optimization of RIR synthesis and distance estimation, enabling end-to-end adaptive fine-tuning. Evaluated on two tasks in the ICASSP 2025 Generative Data Augmentation Workshop, our method achieves significant improvements—reducing average RIR modeling error by 23.6% and improving speaker distance estimation accuracy by 18.4% relatively. The approach establishes a new paradigm for geometry-aware acoustic modeling under low-resource conditions.

Technology Category

• Natural Language Processing: Generation
• Machine Learning: Deep Generative Models & Autoencoders
• Cognitive Modeling & Cognitive Systems: Affective Computing

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

This report details MERL's system for room impulse response (RIR) estimation submitted to the Generative Data Augmentation Workshop at ICASSP 2025 for Augmenting RIR Data (Task 1) and Improving Speaker Distance Estimation (Task 2). We first pre-train a neural acoustic field conditioned by room geometry on an external large-scale dataset in which pairs of RIRs and the geometries are provided. The neural acoustic field is then adapted to each target room by using the enrollment data, where we leverage either the provided room geometries or geometries retrieved from the external dataset, depending on availability. Lastly, we predict the RIRs for each pair of source and receiver locations specified by Task 1, and use these RIRs to train the speaker distance estimation model in Task 2.

Problem

• Estimating room impulse responses using neural acoustic fields
• Improving speaker distance estimation with augmented RIR data
• Adapting pre-trained models to target rooms using enrollment data

Innovation

• Pre-train neural acoustic field with room geometry
• Adapt model using enrollment and retrieved geometries
• Predict RIRs for source-receiver pairs and distance estimation