Backdooring Acoustic Foundation Models for Physically Realizable Triggers

2026-10-07

AI Summary

This study addresses the insufficient research on backdoor security in acoustic foundation models by proposing the FAB attack framework. Without requiring access to pre-training data, this method implants backdoors using covert, synchronization-independent, and cross-task physically realizable triggers, thereby compromising downstream tasks while preserving the model's normal performance. The effectiveness of the proposed approach is validated through adversarial example generation, fine-tuning robustness testing, and a multi-task evaluation framework across two mainstream models and nine tasks. Experimental results demonstrate that FAB can withstand prevalent defense mechanisms, revealing significant security vulnerabilities inherent in acoustic foundation models.

Abstract

Acoustic foundation models (AFMs) have democratized acoustic applications, enabling powerful models for tasks ranging from speech recognition to speaker verification with minimal resources. However, the security of applications based on AFMs remains largely underexplored. Our work addresses this gap by proposing the Foundation Acoustic model Backdoor (FAB) attack, demonstrating that state-of-the-art AFMs are susceptible to backdooring under practical settings. Despite making minimal assumptions about adversary capabilities (e.g., no access to pre-training data), we show that FAB preserves benign performance while inducing backdoors that survive fine-tuning and cause significant degradation across diverse downstream tasks when activated. Notably, FAB utilizes task-agnostic, physically realizable, inconspicuous, and sync-free triggers (e.g., a background siren). We evaluate FAB using two leading AFMs, nine downstream tasks, and four different triggers. We further demonstrate its effectiveness against established defenses and across both digital and physical domains. While extensive end-to-end fine-tuning can mitigate FAB, such a defense is resource-intensive and task-specific. Our work highlights critical risks to AFMs and calls for advanced defenses.

Problem

• Acoustic Foundation Models
• Backdoor Attack
• Security
• Physically Realizable Triggers

Innovation

• Acoustic Foundation Models
• Backdoor Attack
• Physically Realizable Triggers
• Fine-tuning Robustness
• Adversarial Machine Learning