A bilingual AI audiologist built through rubric-guided playbook induction outperforms human audiologists in a blinded evaluation of simulated cases

2026-09-26

AI Summary

This study addresses the scarcity of real-world audiology cases and the unstructured nature of expert consultations by developing a bilingual AI audiologist agent. Methodologically, it introduces a novel rule-guided strategy induction paradigm that eliminates the need for model fine-tuning, integrating multimodal audiogram parsing with retrieval-augmented generation to enable efficient agent construction in data-scarce medical scenarios. In a double-blind evaluation involving 58 cases, the proposed system comprehensively outperformed human experts, achieving a 100% win rate with a Cohen’s d of 1.84. These results validate its exceptional performance in bilingual clinical consultations and establish a new paradigm for deploying specialized AI applications in resource-constrained domains.

Abstract

Audiology consultation requires structured history-taking, audiometric interpretation and patient-centred communication, yet real-world case material is scarce. We present a bilingual AI audiologist pairing a general-purpose large language model with rubric-guided playbook induction, multimodal audiogram interpretation and retrieval-augmented grounding, without fine-tuning the language-model backbone. Using a 21-item rubric and an AI patient simulator, we induced a 19-rule consultation policy from 73 training cases (43 English, 30 Chinese) and evaluated the system on 58 independent simulated cases (30 Chinese, 28 English) in a pre-specified, source-blinded comparison with 17 practising audiologists. The AI audiologist outperformed human audiologists on every case (58/58; mean paired $\Delta$ = +1.35 on a 5-point composite, Cohen's d = 1.84, $P = 4.5 \times 10^{-20}$), on 20 of 21 rubric items and in both languages. Component ablation identified the playbook as the largest contributor, offering a practical route to specialist consultation agents in low-data medical domains.

Problem

• Audiology consultation
• Bilingual AI audiologist
• Low-data medical domains
• Specialist consultation agents

Innovation

• Rubric-guided playbook induction
• Retrieval-augmented grounding
• Multimodal audiogram interpretation
• Low-data medical AI
• Bilingual AI audiologist