Engineering MultiQueues: Fast Relaxed Concurrent Priority Queues

2021-07-03

Embedded Systems and Applications

AI Summary

Traditional concurrent priority queues often become performance bottlenecks under parallel workloads, limiting throughput and increasing latency. This paper proposes MultiQueues—a high-throughput, low-latency concurrent priority queue with relaxed semantics. Methodologically, it employs a multi-sharded queue architecture, triple batching (for insertions, deletions, and load balancing), and a novel wait-free locking technique to enable lock-free transformation from sequential heap structures into efficient concurrent relaxed ones. It further introduces two quantitative metrics—rank error and delay—to rigorously characterize relaxation quality. Experimental evaluation on representative workloads—including online scheduling and discrete-event simulation—demonstrates 3–10× higher throughput over state-of-the-art designs, while maintaining bounded rank error and stable, low latency. MultiQueues thus achieves superior scalability and predictability without sacrificing practicality.

Technology Category

• Planning, Routing, and Scheduling: Optimization of Spatio-temporal Systems
• Data Mining & Knowledge Management: Scalability, Parallel & Distributed Systems
• Multiagent Systems: Mechanism Design

Application Category

• Security and Privacy: Large-scale security measurements
• Systems and Infrastructure for Web, Mobile and WoT: Web performance, measurement, and characterization
• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines

Abstract

Priority queues with parallel access are an attractive data structure for applications like prioritized online scheduling, discrete event simulation, or greedy algorithms. However, a classical priority queue constitutes a severe bottleneck in this context, leading to very small throughput. Hence, there has been significant interest in concurrent priority queues with relaxed semantics. We investigate the complementary quality criteria rank error (how close are deleted elements to the global minimum) and delay (for each element x, how many elements with lower priority are deleted before x). In this paper, we introduce MultiQueues as a natural approach to relaxed priority queues based on multiple sequential priority queues. Their naturally high theoretical scalability is further enhanced by using three orthogonal ways of batching operations on the sequential queues. Experiments indicate that MultiQueues present a very good performance-quality tradeoff and considerably outperform competing approaches in at least one of these aspects. We employ a seemingly paradoxical technique of"wait-free locking"that might be of more general interest to convert sequential data structures to relaxed concurrent data structures.

Problem

• Addressing low throughput in parallel priority queues
• Improving scalability via buffering and cache optimization
• Balancing throughput and quality in relaxed semantics

Innovation

• Uses multiple sequential priority queues
• Enhances scalability via buffering and batching
• Employs wait-free locking technique