AudioProcessBench: Benchmark for Identifying Process Errors in Audio-Grounded Reasoning

2026-06-07

AI Summary

Existing evaluation benchmarks struggle to assess step-level errors in the reasoning processes of large audio language models. To address this gap, this work introduces AudioProcessBench, the first fine-grained benchmark specifically designed for evaluating audio reasoning. Constructed from reasoning trajectories generated by six audio and multimodal language models, the benchmark features human-annotated step segmentation, binary correctness judgments, and fine-grained categorization of audio-specific error types. AudioProcessBench supports three evaluation paradigms: step-wise correctness identification, error-type-conditioned detection, and reasoning-chain aggregation, thereby providing a systematic testbed for developing audio reasoning verifiers, process-based reward models, and reliable multimodal reasoning systems.

Abstract

Large audio-language models (LALMs) increasingly use explicit reasoning traces for complex audio understanding, yet the evaluation of reasoning quality remains underexplored. Although process-level benchmarks for process reward models (PRMs) have advanced reasoning evaluation in text and multi-modal domains, comparable evaluation for audio reasoning remains limited. In this paper, we present AudioProcessBench, a comprehensive benchmark for step-level process error identification in audio reasoning. AudioProcessBench contains diverse reasoning traces generated by 6 audio and omni language models. Each trace is segmented into discrete reasoning steps and annotated with binary step correctness and fine-grained error types. Our benchmark evaluates models under three complementary paradigms: (1) step correctness identification, (2) error-type-conditioned detection for diagnosing audio-specific verifier capacities, and (3) chain-level aggregation, where verifiers select or aggregate among multiple reasoning traces for the same question. This design enables a systematic analysis of whether current models can detect process errors, whether their weaknesses differ across audio-specific error types, and whether process verification translates into improved answer selection. AudioProcessBench provides a testbed for future research on audio reasoning verifiers, process reward models, and reliable omni-modal reasoning.

Problem

• audio reasoning
• process error identification
• reasoning trace evaluation
• process reward models
• audio-language models

Innovation

• audio reasoning
• process error identification
• process reward models
• step-level evaluation
• multi-modal verification