psychoacoustic modeling

Existing music question-answering benchmarks (e.g., MuchoMusic) severely overestimate models’ audio perception capabilities: pure text-based LLMs or models fed Gaussian noise achieve high accuracy, indicating that these benchmarks primarily evaluate linguistic reasoning—not auditory understanding. Method: We propose RUListening, a novel framework introducing the quantifiable Perceptual Index (PI) to measure the extent to which a question genuinely requires audio input; we further design perceptual-aware data filtering and adversarial distractor synthesis—both grounded in LLM log-probability distributions—to reconstruct a rigorous subset of MuchoMusic. Results: On the revised benchmark, text-only LLMs drop to near-random accuracy (~20%), while listening-augmented LLMs (LALMs) exhibit significant performance degradation under audio corruption, enabling, for the first time, effective and trustworthy evaluation of true audio perception capability.

This study investigates how stereo processing—specifically Mid/Side versus Left/Right encoding—affects subjective audio quality perception, and evaluates the predictive accuracy of mainstream objective metrics (e.g., PEAQ, ITU-R BS.1387, DNSMOS) under spatial distortions. Leveraging the ODAQ stereo extension dataset with corresponding Mean Opinion Scores (MOS), we conduct time–frequency domain metric comparisons and statistical modeling. Our analysis quantitatively reveals, for the first time, the critical interplay between bottom-up auditory mechanisms and top-down contextual factors in stereo quality prediction. Results show that timbre-oriented metrics remain robust under simple distortions but degrade significantly under spatial distortions; current models exhibit systematic bias due to their neglect of spatial dimensions. We propose a novel three-dimensional perceptual evaluation paradigm integrating temporal, spectral, and spatial cues—providing both theoretical foundation and methodological support for next-generation audio quality metrics.

Personalized HRTF modeling commonly relies on latent spaces optimized for spectral reconstruction, yet these spaces lack alignment with human auditory perception, leading to perceptually distorted spatial audio. Method: This work first systematically evaluates the correlation between existing HRTF latent representations and objective psychoacoustic metrics—including interaural level/time difference (ILD/ITD) sensitivity and spectral difference perception models. We then propose a perception-guided HRTF embedding framework that integrates auditory perceptual metrics into deep learning: (i) a perception-aware metric loss function based on perceptual similarity, and (ii) metric multidimensional scaling (MMDS) to align the geometry of the latent space with empirically grounded perceptual relationships. Contribution/Results: Experiments demonstrate that our method significantly outperforms conventional approaches in both HRTF reconstruction accuracy and perceptual consistency, yielding substantial improvements in the realism and auditory naturalness of personalized spatial audio.

This study addresses the misalignment between conventional optimization objectives and human auditory preferences in generative speech enhancement. We propose a preference-aligned method leveraging language model architectures, introducing Direct Preference Optimization (DPO) to speech enhancement for the first time. To enable scalable, annotation-free preference supervision, we adopt UTMOS as a proxy perceptual feedback model. Unlike standard mean-squared-error or likelihood-based training paradigms, our approach directly optimizes for human subjective quality preferences. Experiments on the DNS2020 test set demonstrate that the proposed method achieves up to a 56% relative improvement over baseline models in objective metrics—including PESQ, STOI, and VISQOL—while markedly enhancing speech naturalness and intelligibility. These results validate the effectiveness and generalizability of perception-driven optimization in generative speech enhancement.

To address the longstanding reliance on single-task models and the lack of generalizable representations in computer audition, this paper proposes a systematic framework for constructing Auditory Foundation Models (AFMs). Methodologically, it establishes the core paradigm of AFMs for the first time, integrating unified multi-task modeling, cross-modal (audio–text) aligned representation learning, and instruction-driven human–machine interaction. Technically, the framework encompasses large-scale audio–text contrastive pretraining, multi-task prompt tuning, and self-supervised audio modeling. Experiments demonstrate that the proposed AFM achieves substantial performance gains across 10+ downstream tasks—including automatic speech recognition, sound source separation, and environmental sound classification—while enabling zero-shot transfer and open-domain speech understanding. This work advances computer audition toward generality, multi-task synergy, and natural human–machine interaction.

This study addresses the limited capability of large audio language models in fundamental auditory perception—such as pitch, loudness, and spatial location—where their performance often approaches random guessing and fails to match human superiority in comparative tasks. To systematically evaluate this gap, the authors propose SonicBench, the first psychophysics-based benchmark integrating controllable audio generation, a dual-task paradigm of identification and comparison, linear probing analysis, and controlled experiments. Findings reveal that frozen audio encoders already capture physical auditory cues effectively (accuracy ≥60%), indicating that the primary bottleneck lies not in perceptual encoding but in subsequent alignment and decoding stages. This work provides the first systematic characterization of the boundaries of audio language models in basic auditory perception.

This work addresses the susceptibility of large audio language models to hallucinations caused by linguistic priors overpowering acoustic evidence. To mitigate this, the authors propose a task- and sample-adaptive perturbation selection mechanism within a contrastive decoding framework. Leveraging a structured audio perturbation bank spanning temporal, spectral, frequency, and amplitude domains, the method dynamically selects optimal negative-sample perturbation strategies and employs a lightweight selector for efficient routing. The approach yields a 4.3% absolute improvement in accuracy on existence tasks and significantly boosts performance on temporal tasks from 74.7% to 81.4%. Furthermore, the study validates the efficacy of binary-constrained prompts, underscoring the critical role of adaptive perturbation strategies in alleviating hallucinations in audio language models.

Subjective listening tests remain the bottleneck for evaluating speech quality of neural audio codecs at low bitrates. Method: We systematically benchmark mainstream objective metrics—including PESQ, STOI, and DNSMOS—against human perception using standardized MUSHRA subjective test results, quantifying their correlation with mean opinion scores via Pearson’s correlation coefficient. Contribution/Results: Traditional metrics (e.g., PESQ) exhibit markedly degraded performance under neural codec distortions, whereas DNSMOS and novel time-frequency domain metrics achieve superior correlation (r > 0.85). We are the first to characterize differential sensitivity of objective metrics to neural-specific artifacts—such as spectral smearing and temporal aliasing—and to propose an empirically grounded, optimized metric combination with clearly defined applicability boundaries for neural audio codecs. This work provides evidence-based guidelines for automated, reproducible speech quality assessment in neural codec development and evaluation.

This work addresses the vulnerability of post-hoc explainability methods in audio deepfake detection, where explanations can be manipulated without altering model predictions, thereby undermining the reliability of explainable AI (XAI). For the first time, explanation manipulation is extended to the audio domain through a psychoacoustically constrained framework that systematically distorts attribution heatmaps by optimizing imperceptible perturbations while preserving the original prediction. Integrating psychoacoustic modeling, imperceptible perturbation generation, and attribution alignment evaluation, the proposed method demonstrates significant manipulability of attributions across multiple state-of-the-art audio deepfake detectors. These findings expose the fragility of current XAI approaches in audio tasks and introduce a novel evaluation dimension for building trustworthy audio AI systems.

This work addresses the concern that high performance of current audio-language models on standard benchmarks may stem from reliance on textual priors rather than genuine utilization of acoustic signals for auditory comprehension. To investigate this, the authors propose a diagnostic framework that systematically quantifies the extent to which models actually depend on audio inputs. Through ablation studies and segment-level audio analysis, they evaluate eight prominent audio-language models across three major benchmarks. Their findings reveal that models retain 60–72% of their original accuracy even without any audio input, and only 3.0–4.2% of questions genuinely require the full audio signal—most tasks can be solved using brief local segments. These results challenge prevailing evaluation paradigms and expose a critical limitation: existing benchmarks inadequately assess true auditory reasoning capabilities.