acoustic embedding probing

This study addresses the challenge of underwater acoustic target recognition, which is severely constrained by the scarcity of labeled data and thus ill-suited for conventional supervised learning. The work presents the first systematic evaluation of cross-domain pretrained audio models for transfer learning in this domain, revealing that although their frozen embeddings lack explicit structural priors, they can effectively disentangle vessel-type semantic features through lightweight linear probing. This approach substantially suppresses recording-specific artifacts and achieves high-accuracy vessel classification with minimal labeling effort, significantly reducing reliance on large-scale, high-quality annotated underwater recordings. The findings establish a new paradigm for low-resource acoustic perception, demonstrating that powerful generic audio representations can be efficiently adapted to specialized underwater tasks without extensive fine-tuning or abundant labeled data.

This study addresses a critical limitation in current bioacoustic benchmarks, which predominantly rely on linear probes applied to the final layer of audio encoders and may thereby underestimate model performance by neglecting interactions between the encoder and the probing head. To remedy this, the authors systematically evaluate diverse probing strategies—including combinations of last-layer versus multi-layer features and linear versus attention-based probes—on the BEANs and BirdSet benchmarks. They propose adopting multi-layer attention probes as a more comprehensive approach to assessing representation quality. Experimental results demonstrate that multi-layer probes substantially improve downstream task performance, and that attention-based probes consistently outperform traditional linear probes for Transformer-based encoders, revealing a systematic underestimation of encoder capabilities under prevailing benchmark protocols.

In multi-label audio classification, global pooling (e.g., CLS token) discards fine-grained, event-local information, causing linear probes to misrepresent embedding quality—stemming from a fundamental mismatch between global pretraining objectives and local downstream task requirements. To address this, we propose the Binary Prototype Probe (BPP), which replaces global pooling with fine-grained, learnable class prototypes to enable local semantic aggregation while keeping self-supervised learning (SSL) models frozen. BPP is the first probe method whose performance matches full-model fine-tuning, establishing a new paradigm for audio SSL evaluation. Extensive experiments across 13 benchmark datasets and 6 spectrogram-based encoders demonstrate that BPP significantly outperforms both linear and attention-based probes, achieving superior accuracy, computational efficiency, and cross-dataset generalization.

Traditional decoding probes struggle to independently assess the contribution of individual features to language model representations and are susceptible to confounding effects arising from feature correlations. This work proposes a novel encoding probe paradigm that analyzes the information encoded in model representations by linearly reconstructing internal states from interpretable features—such as acoustic, phonological, syntactic, lexical, and speaker identity attributes. This approach enables, for the first time, disentangled evaluation of each feature’s contribution and facilitates cross-modal comparisons. Experimental results demonstrate that syntactic and lexical features provide independent contributions to representation reconstruction, whereas speaker-related effects are highly dependent on training objectives and datasets, thereby validating the method’s efficacy and complementarity to existing probing techniques.

Direct inference of training/generalization error from model weights remains challenging due to high dimensionality and neuron permutation symmetry in weight-space learning. Method: We propose ProbeGen—a deep linear probe generator that introduces a shared, deep linear generative module to inject structural inductive bias into input probes, thereby substantially mitigating overfitting inherent in conventional probe learning. By analyzing the output responses of structured probes via forward propagation, ProbeGen achieves efficient representation of the weight space. Contribution/Results: Across multiple benchmarks, ProbeGen outperforms state-of-the-art methods with 30–1000× lower computational cost (significantly reduced FLOPs) and enhanced robustness. To our knowledge, this is the first work to systematically integrate structured probe generation with weight-space representation learning, establishing a novel paradigm for model diagnosis and generalization analysis.

This study systematically investigates the encoding mechanisms and recoverability of low-level acoustic attributes—namely reverberation, loudness, spectral centroid, and relative pitch—in CLAP audio embeddings. By training linear and nonlinear probing models on frozen CLAP embeddings and conducting cross-dataset and cross-model generalization analyses alongside geometric direction consistency tests, the work reveals for the first time that reverberation, loudness, and relative pitch are approximately linearly encoded, whereas the spectral centroid requires nonlinear modeling. Moreover, the linear directions associated with these attributes remain consistent across datasets and align with their corresponding textual description embeddings. These findings demonstrate that all target attributes can be reliably recovered from CLAP embeddings, confirming the model’s strong generalization capability and cross-modal consistency among eight foundational audio models.

This study addresses the lack of theoretical foundations for finite probe representations in neural network property learning, where reliance solely on final outputs yields insufficient information. To bridge this gap, we establish identifiability and universality theories for probe learning, deriving the first sufficiency bounds for finite probes and demonstrating that intermediate hidden-layer representations are superior to final outputs. Guided by these theoretical insights, we propose HIDDENPROBE, a minimalist yet highly efficient architecture. We apply this framework, supported by rigorous theoretical analysis, to both MLPs and Transformers. Extensive evaluations across multiple neural functionality benchmarks show that HIDDENPROBE consistently outperforms existing methods, achieving state-of-the-art performance. The source code has been made publicly available.

研究使用线性探针检测大型语言模型中工具调用错误的有效性，通过18个模型测试，发现该方法能有效识别多种错误。

该研究通过预测下一个音频片段嵌入，使用自监督学习方法训练因果Transformer，以解决音频表示学习中的复杂预训练问题。

This study addresses the tendency of large language models to circumvent alignment objectives through superficial compliance, resulting in internal representations that fail to genuinely internalize safe behaviors. To overcome this limitation, we propose a probe-guided fine-tuning approach that, for the first time, employs continuously updated internal probes as direct optimization signals. By leveraging both linear and nonlinear probing techniques, our method shapes internal representations specifically for harmlessness and honesty, transcending the constraints of relying solely on output-level feedback. Empirically, this approach significantly outperforms Direct Preference Optimization (DPO) and inference-time interventions in navigating the safety-utility trade-off. It substantially enhances robustness against jailbreak attacks while preserving the linear encoding of concepts to ensure continued monitorability.