Cost-Aware Best-LLM Identification using Dueling Feedback

2026-09-24

AI Summary

This study addresses the problem of efficiently identifying the optimal model from an ensemble of large language models with heterogeneous query costs based on pairwise comparison feedback. To this end, it introduces a dueling feedback mechanism into a cost-aware multi-armed bandit framework and, under the Condorcet winner assumption, proposes a Track-and-Stop algorithm incorporating confidence constraints. The primary contribution lies in providing the first unified formulation of dueling feedback and heterogeneous sampling costs, along with theoretical guarantees that the algorithm asymptotically achieves optimal cost as the error probability vanishes. Experiments on both synthetic and real-world datasets demonstrate that the proposed method consistently outperforms conventional cost-agnostic algorithms and their extensions.

Abstract

Inspired by the problem of identifying the best model from a collection of large language models (LLMs) with heterogeneous querying costs, we formulate and analyse a variant of the multi-armed bandit (MAB) with (i) dueling feedback, where pairwise comparisons between model responses provide robust preference signals, and (ii) heterogeneous sampling costs, reflecting the differing costs of querying different LLMs. Assuming the existence of a Condorcet winner, a condition we empirically validate across multiple real-world datasets, we propose a Track-and-Stop style algorithm for best-arm identification with prescribed confidence. We prove that the algorithm almost surely achieves the asymptotically optimal cost as the error tends to zero. Finally, we extensively evaluate our approach on both synthetic and real-world instances, demonstrating consistent improvements over classical cost-unaware algorithms and their cost-aware extensions.

Problem

• Best-LLM Identification
• Dueling Feedback
• Heterogeneous Sampling Costs
• Multi-Armed Bandit

Innovation

• Dueling Bandits
• Best-Arm Identification
• Cost-Aware Sampling
• Large Language Models
• Track-and-Stop