Problem
Research questions and friction points this paper is trying to address.
Spoken Language Understanding
open-domain tasks
in-context learning
semantic extraction
rule ambiguity
Spoken Function Calling: A New Perspective on Spoken Language Understanding for Large Audio Language Models
🤖 AI Summary
This work addresses the limitations of traditional spoken language understanding (SLU) in open-domain scenarios, where ambiguous linguistic patterns and limited contextual learning hinder precise semantic parsing. To overcome these challenges, the paper introduces a novel paradigm termed Spoken Function Calling (SFC), which for the first time integrates structured function calling into SLU, thereby transcending the constraints of closed-set classification and enabling accurate modeling of open-domain semantics. The authors construct the SFC-Bench dataset via a multi-agent system and further enhance semantic extraction through post-training optimization leveraging both large language models (LLMs) and large audio language models (LALMs). Experimental results demonstrate that SFC consistently achieves substantial improvements over conventional SLU approaches across multiple model architectures.
📝 Abstract
Spoken Language Understanding (SLU) is the core component of task-oriented dialogue systems and a pivotal link in achieving seamless human-agent interaction. While traditional SLU can effectively extract user semantics for closed-set tasks after in-domain supervised fine-tuning, it faces significant challenges in leveraging in-context learning for open-domain tasks due to its ambiguous rule definitions. This work proposes Spoken Function Calling (SFC), a novel semantic understanding perspective that optimizes semantic understanding with structured rule definitions, to evolve beyond traditional closed-set SLU. Specifically, we curate and extend a suite of spoken functions based on traditional SLU datasets, construct a multi-agent system to synthesize the SFC-Bench dataset, evaluate the performance of Large Language Models (LLMs) and Large Audio Language Models (LALMs), and enhance the SFC capabilities of LALMs through post-training. Experiments demonstrate that SFC outperforms traditional SLU, substantially enhancing the semantic extraction accuracy for LLMs and LALMs.
Innovation
Methods, ideas, or system contributions that make the work stand out.
Spoken Function Calling
Large Audio Language Models
Structured Semantic Understanding
In-context Learning
SFC-Bench
🔎 Similar Papers
No similar papers found.
💼 Related Jobs
