Ambisonics Binaural Rendering via Masked Magnitude Least Squares

2025-01-30

AI Summary

In low-order Ambisonics binaural rendering, high-frequency spatial cues—particularly pinna-filtering notches in head-related transfer functions (HRTFs)—are often lost, degrading sound source localization accuracy. To address this, we propose Masked Magnitude Least Squares (MMLS), a psychoacoustically motivated optimization framework that introduces a learnable spectral-spatial weighting mask within a magnitude least-squares objective. Unlike conventional approaches, MMLS jointly optimizes Ambisonics coefficients and the mask via neural networks, explicitly preserving critical HRTF notch structures in the median plane. This enables faithful reconstruction of high-frequency localization cues while maintaining overall spectral fidelity. Experiments demonstrate substantial improvements in localization accuracy under low-order rendering conditions, with negligible magnitude reconstruction error. The method establishes a new paradigm for cost-effective, high-fidelity 3D audio rendering by bridging perceptual requirements with computationally efficient signal modeling.

Technology Category

• Machine Learning: Multimodal Learning
• Cognitive Modeling & Cognitive Systems: Neural Spike Coding
• Search and Optimization: Mixed Discrete/Continuous Search

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Ambisonics rendering has become an integral part of 3D audio for headphones. It works well with existing recording hardware, the processing cost is mostly independent of the number of sound sources, and it elegantly allows for rotating the scene and listener. One challenge in Ambisonics headphone rendering is to find a perceptually well behaved low-order representation of the Head-Related Transfer Functions (HRTFs) that are contained in the rendering pipe-line. Low-order rendering is of interest, when working with microphone arrays containing only a few sensors, or for reducing the bandwidth for signal transmission. Magnitude Least Squares rendering became the de facto standard for this, which discards high-frequency interaural phase information in favor of reducing magnitude errors. Building upon this idea, we suggest Masked Magnitude Least Squares, which optimized the Ambisonics coefficients with a neural network and employs a spatio-spectral weighting mask to control the accuracy of the magnitude reconstruction. In the tested case, the weighting mask helped to maintain high-frequency notches in the low-order HRTFs and improved the modeled median plane localization performance in comparison to MagLS, while only marginally affecting the overall accuracy of the magnitude reconstruction.

Problem

• 3D sound quality
• HRTF processing
• cost-effectiveness

Innovation

• Masked Amplitude Least Squares
• High-frequency Detail Enhancement
• Efficient 3D Audio Rendering