🤖 AI Summary
This paper addresses the degradation of early-learning performance in queueing systems caused by parameter uncertainty. To quantify this issue, it introduces the *Queue Learning Cost* (CLQ)—a novel metric capturing the maximum transient growth in time-averaged queue length under unknown system parameters. Methodologically, the work establishes a unified analytical framework integrating Lyapunov stability theory with multi-armed bandit analysis, leveraging asymptotic and non-asymptotic probabilistic tools, stochastic bandwidth modeling, and queueing network techniques. The CLQ is fully characterized for single-queue, multi-server systems and extended to multi-queue, multi-server configurations and general queueing networks. The resulting theory provides the first unified, verifiable early-time performance guarantees for a broad class of learning-based scheduling algorithms—thereby filling a fundamental theoretical gap in the transient analysis of learning-controlled queueing systems.
📝 Abstract
Queueing systems are widely applicable stochastic models with use cases in communication networks, healthcare, service systems, etc. Although their optimal control has been extensively studied, most existing approaches assume perfect knowledge of the system parameters. Of course, this assumption rarely holds in practice where there is parameter uncertainty, thus motivating a recent line of work on bandit learning for queueing systems. This nascent stream of research focuses on the asymptotic performance of the proposed algorithms. In this paper, we argue that an asymptotic metric, which focuses on late-stage performance, is insufficient to capture the intrinsic statistical complexity of learning in queueing systems which typically occurs in the early stage. Instead, we propose the Cost of Learning in Queueing (CLQ), a new metric that quantifies the maximum increase in time-averaged queue length caused by parameter uncertainty. We characterize the CLQ of a single queue multi-server system, and then extend these results to multi-queue multi-server systems and networks of queues. In establishing our results, we propose a unified analysis framework for CLQ that bridges Lyapunov and bandit analysis, provides guarantees for a wide range of algorithms, and could be of independent interest.
