HyPASE: Hyperbolic Geometry for Parameter-Efficient Speech Emotion Fine-Tuning Framework for Large Audio-Language Models

2026-08-04

AI Summary

This work addresses the limitations of existing parameter-efficient fine-tuning (PEFT) methods, which operate in Euclidean space and struggle to capture the multi-granular and hierarchical nature of speech emotion. To overcome this, we propose HyPASE, a novel framework that introduces hyperbolic geometry into the fine-tuning of large audio-language models for emotion recognition. HyPASE leverages the Poincaré ball model to explicitly encode semantic granularity via radius and integrates a Hyperbolic Geometry Adapter (HGA) with an Emotion-aware Multi-capacity Cross-modal Aggregator (EMCA) to enable efficient, structure-aware adaptation. Experiments demonstrate that HyPASE consistently outperforms Euclidean PEFT baselines on MELD and significantly improves unweighted accuracy on IEMOCAP, particularly excelling under class-imbalanced conditions while exhibiting strong zero-shot cross-dataset generalization.

Abstract

Large Audio-Language Models (LALMs) excel at general speech understanding; however, adapting them to fine-grained tasks like Speech Emotion Recognition (SER) remains a significant bottleneck. Current Parameter-Efficient Fine-Tuning (PEFT) methods typically operate in flat Euclidean space, and this geometry fails to capture the multi-granularity nature of emotion cues, which range from low-level prosody to high-level semantics. To address this, we propose HyPASE, a hyperbolic PEFT framework for LALM-based SER. HyPASE leverages the Poincare ball model, using the hyperbolic radius as an explicit proxy for representational granularity. The framework consists of two core components: a Hyperbolic Geometric Adapter (HGA) for layer-adaptive weight modulation, and an Emotion-aware Multi-capacity Cross-modal Aggregator (EMCA) that compresses multi-scale features into compact audio prefixes. Empirical results on standard benchmarks show that HyPASE outperforms Euclidean PEFT baselines across all metrics on MELD and achieves a notable Unweighted Accuracy gain on IEMOCAP, particularly in class-imbalanced emotion recognition, with the accompanying slight Weighted Accuracy trade-off reflecting hyperbolic space's geometric prioritization of minority-class representations; furthermore, HyPASE achieves robust zero-shot cross-dataset generalization within a constrained parameter budget. By grounding the adaptation process in hyperbolic geometry, HyPASE offers a highly efficient path for LALM fine-tuning.

Problem

• Speech Emotion Recognition
• Parameter-Efficient Fine-Tuning
• Large Audio-Language Models
• Hyperbolic Geometry
• Multi-granularity Representation

Innovation

• Hyperbolic Geometry
• Parameter-Efficient Fine-Tuning
• Speech Emotion Recognition
• Large Audio-Language Models
• Multi-granularity Representation