ArXiv

Spoken Function Calling: A New Perspective on Spoken Language Understanding for Large Audio Language Models

2026-08-05 · 17:50 UTC ·Yuezhang Peng, Yuxin Liu, Changfeng Gao... ·1 min read
Authors
Yuezhang Peng, Yuxin Liu, Changfeng Gao...
Categories
cs.CL, cs.MM
arXiv
https://arxiv.org/abs/2608.05126v1
PDF
https://arxiv.org/pdf/2608.05126v1

Brief

Spoken Function Calling (SFC) reframes spoken language understanding by replacing ambiguous closed-set SLU labels with structured function definitions to support in-context learning for open-domain tasks. The authors curate spoken functions, synthesize the SFC-Bench dataset with a multi-agent pipeline, evaluate LLMs and LALMs, and apply post-training to boost LALM performance; experiments show notable gains in semantic extraction accuracy.

Why it matters

Introduces Spoken Function Calling (SFC), a structured-rule semantic representation designed to enable in-context learning for open-domain spoken language understanding, addressing limitations of traditional closed-set SLU.

Key details

  • Authors curated and extended spoken functions and synthesized the SFC-Bench dataset using a multi-agent system; they evaluated both LLMs and Large Audio Language Models (LALMs) and improved LALM SFC capabilities via post-training.
  • Experiments report that SFC outperforms traditional SLU by substantially enhancing semantic extraction accuracy for LLMs and LALMs; paper is on arXiv (2608.05126v1) and listed for ACM Multimedia 2026 (posted 2026-08-05).
Source evidence

Abstract

Spoken Language Understanding (SLU) is the core component of task-oriented dialogue systems and a pivotal link in achieving seamless human-agent interaction. While traditional SLU can effectively extract user semantics for closed-set tasks after in-domain supervised fine-tuning, it faces significant challenges in leveraging in-context learning for open-domain tasks due to its ambiguous rule definitions. This work proposes Spoken Function Calling (SFC), a novel semantic understanding perspective that optimizes semantic understanding with structured rule definitions, to evolve beyond traditional closed-set SLU. Specifically, we curate and extend a suite of spoken functions based on traditional SLU datasets, construct a multi-agent system to synthesize the SFC-Bench dataset, evaluate the performance of Large Language Models (LLMs) and Large Audio Language Models (LALMs), and enhance the SFC capabilities of LALMs through post-training. Experiments demonstrate that SFC outperforms traditional SLU, substantially enhancing the semantic extraction accuracy for LLMs and LALMs.

Comment: ACM Multimedia 2026