Improving Zero-Shot Phonetic Classification through Language-Agnostic Articulatory Features

2026-07-26

AI Summary

This work addresses the limitations of existing grapheme-to-phoneme (G2P)-based speech transcription models in zero-shot settings, which struggle to faithfully capture phonological structure and exhibit poor accuracy on unseen languages or rare phonemes—such as Mandarin aspirated stops and Japanese moraic nasals. To overcome this, the authors propose a novel zero-shot phonological classification approach that leverages continuous articulatory feature (AF) vectors instead of discrete IPA symbols. They introduce frame-level and segment-level temporal aggregation strategies tailored to phonological characteristics and demonstrate, for the first time, the efficacy of language-agnostic AFs in this task. Their findings reveal that optimal temporal aggregation must be dynamically adapted to the target phonology: frame-level aggregation excels for aspirated consonants, while segment-level aggregation substantially improves moraic nasal recognition. The method significantly outperforms existing IPA-based approaches in generalization performance.

Abstract

Recent Phonetic Foundation Models (PFMs) for Speech-to-IPA transcription rely on Grapheme-to-Phoneme (G2P) labels, but the phoneme labels are not necessarily phonetically faithful. To investigate this issue, we evaluate zero-shot phonetic classification on Chinese aspiration and Japanese moraic nasals. A PFM trained on G2P-labeled data excluding these two languages yields poor accuracy on both tasks, showing that multilingual coverage with discrete IPA tokens is not sufficient for unseen settings. To overcome this limitation, we propose a classification method based on continuous Articulatory Feature (AF) vectors extracted from each frame. This AF-based approach outperforms discrete token-based methods, particularly for rare phones. We further show that it is crucial to adopt the optimal temporal aggregation of AF vectors for the target distinction: single-frame classification is best for aspiration, while segmental classification substantially improves nasal classification.

Problem

• Zero-Shot Phonetic Classification
• Phonetic Foundation Models
• Articulatory Features
• IPA Transcription
• Cross-lingual Generalization

Innovation

• Articulatory Features
• Zero-Shot Phonetic Classification
• Phonetic Foundation Models
• Continuous Representation
• Temporal Aggregation