audio editing

Audio editing has long suffered from the lack of high-quality evaluation benchmarks and reliable automated assessment metrics. To address this, we propose an expert-knowledge-driven closed-loop evaluation framework. First, we construct AuditScore—the first subjective evaluation dataset for audio editing—comprising over 6,300 samples annotated with multi-dimensional professional ratings. Second, we train AuditEval, an automatic Mean Opinion Score (MOS) prediction model achieving high accuracy in quality estimation. Third, we leverage AuditEval in a reverse pipeline to filter and refine synthetic data, generating a validated pseudo-parallel dataset of superior quality. This work pioneers the organic integration of expert scoring, automated evaluation, and data curation: it introduces the first task-specific audio editing evaluation model and benchmark dataset, and establishes a “evaluate–feedback–generate” closed-loop paradigm—providing a reproducible, scalable foundation for future research and development in audio editing.

This work addresses the risk of spurious correlations between recording artifacts and target labels in speech datasets, which arise from heterogeneous recording conditions and can lead to overestimated model performance—posing significant safety concerns in high-stakes domains such as healthcare. To detect such confounding factors, the study introduces a novel diagnostic method that leverages non-speech segments within audio recordings to predict target labels, exploiting the metadata implicitly embedded in these silent or non-linguistic regions. By analyzing these non-speech intervals, the approach effectively identifies spurious associations present in both training and test data. The authors further release an open-source toolkit implementing this technique, substantially enhancing the reliability of speech model evaluation and the trustworthiness of real-world deployment.

Existing audio classifiers often rely on non-musical features or spurious correlations, rendering them susceptible to misleading cues. This work introduces, for the first time, the causal inference concepts of sufficiency and necessity into the interpretability of audio classification. By conducting frequency-domain analysis, the study identifies critical subsets of frequency components that are causally responsible for model decisions and develops FreqReX, a tool enabling targeted interventions. Experiments demonstrate that altering just a single frequency bin among 240,000 can flip the classification outcome in 58% of cases, with perturbations remaining nearly imperceptible to human listeners. These findings expose significant model fragility and offer a novel pathway toward more robust and interpretable audio classification systems.

Existing single-sample membership inference (MI) methods fail on large-scale, heterogeneous audio datasets, hindering copyright protection for generative audio models. Method: We propose the first verifiable, audio-domain-specific training data attribution framework—Dataset Inference (DI)—a novel set-level inference paradigm that aggregates multi-artist audio samples to determine whether a given dataset contributed to model training. Our approach jointly leverages gradient and output-statistical features from diffusion and autoregressive audio models, incorporates a multi-sample evidence aggregation mechanism, and integrates contrastive benchmark modeling with statistical significance testing. Contribution/Results: Evaluated on multiple open-source large audio models, DI achieves high inference accuracy (AUC > 0.92), substantially outperforming state-of-the-art MI methods. This work provides the first empirically validated, technically feasible solution for audio content copyright auditing and training-data accountability.

Multilingual speech datasets—particularly for low-resource languages—suffer from pervasive macro-level (e.g., ambiguous dialect boundaries, absence of language planning) and micro-level (e.g., grapheme–phoneme inconsistency) quality deficiencies, severely impeding ASR model training and evaluation. This paper takes Taiwanese Hokkien (nan_tw) as a case study and proposes, for the first time, a dual-track framework integrating sociolinguistic awareness and prospective language planning to embed linguistic governance directly into ASR data curation. Through cross-dataset auditing (Common Voice, FLEURS, VoxPopuli), fieldwork, dialect annotation consistency assessment, and orthographic adaptability testing, we identify significant macro-level risks in 21 of 37 languages examined. The work yields an actionable, linguistically grounded guideline for multilingual speech dataset construction, formally adopted by Hugging Face as the v2.0 community standard.

Existing Foley sound datasets generally suffer from insufficient quality and coarse annotations, hindering data-driven research in classification, retrieval, and synthesis. To address this gap, this work introduces and publicly releases FoleySet—a large-scale Foley dataset comprising 10,000 audio clips meticulously recorded following professional Foley practices. The dataset features a two-tier manual semantic annotation scheme that precisely aligns synchronous sound effects with on-screen human actions, such as footsteps, clothing rustles, and prop manipulations. FoleySet is the first to offer multi-level annotations, standardized formatting, and a permissive Creative Commons license, thereby filling a critical resource void in the field. It provides strong support for Foley-related audio tasks and advances research toward automated audiovisual content production.

Existing evaluation methods for audio captioning struggle to accurately assess the fidelity of multimodal semantics and acoustic attributes in structured audio descriptions. This work proposes the first multi-axis evaluation framework tailored for structured audio captioning, integrating large language model (LLM)-based semantic judgments with deterministic acoustic metrics across five orthogonal dimensions: label sets, descriptive content, logical reasoning, numerical measurements, and spectral contours. The framework incorporates a controlled perturbation protocol to validate its ability to distinguish between semantic preservation and acoustic distortion. Experiments on the AudioCards dataset demonstrate that the proposed approach effectively differentiates semantically consistent paraphrases from genuine errors, significantly outperforming existing methods in both reliability and sensitivity.

This study addresses the limited utility of long-duration child speech recordings in early language development research, hindered by poor cross-corpus compatibility, absence of standardized evaluation protocols, and privacy concerns. To overcome these challenges, the authors propose the first integrated framework that unifies data standardization, reproducible benchmarking, and ethical governance. The framework consolidates 27 open-source child speech datasets, establishes four standardized speech processing benchmark pipelines, and implements a role-based ELSI (Ethical, Legal, and Social Implications) governance system. Validation through a vocal-type classification task demonstrates that the framework substantially enhances model generalizability across languages and recording conditions while ensuring regulatory compliance, thereby achieving a coherent integration of technical rigor and ethical standards.

This work addresses the challenge in audio question answering where models often over-rely on textual priors or are misled by mismatched audio inputs. To mitigate this, the authors propose a diagnostic data curation strategy based on model confusion patterns. By probing model responses under normal, silent, and shuffled audio conditions, they identify and retain samples exhibiting strong audio dependence for fine-tuning. Using the Qwen3-Omni-30B-A3B-Instruct model, the approach integrates counterfactual audio testing, response normalization, and multi-model ensembling. Fine-tuning solely on the curated dataset yields a 67.27% accuracy on the official development set, significantly outperforming a local baseline of 65.90%, thereby demonstrating the method’s effectiveness in enhancing models’ reliance on genuine audio evidence.

This work addresses the vulnerability of post-hoc explainability methods in audio deepfake detection, where explanations can be manipulated without altering model predictions, thereby undermining the reliability of explainable AI (XAI). For the first time, explanation manipulation is extended to the audio domain through a psychoacoustically constrained framework that systematically distorts attribution heatmaps by optimizing imperceptible perturbations while preserving the original prediction. Integrating psychoacoustic modeling, imperceptible perturbation generation, and attribution alignment evaluation, the proposed method demonstrates significant manipulability of attributions across multiple state-of-the-art audio deepfake detectors. These findings expose the fragility of current XAI approaches in audio tasks and introduce a novel evaluation dimension for building trustworthy audio AI systems.