STEM2: A Fast and Space-efficient Data Structure for Exact Multi-Set Membership Query

2026-07-18

AI Summary

This work addresses the challenge of multi-set membership queries, where existing approaches struggle to simultaneously achieve high accuracy, memory efficiency, and efficient dynamic updates—hash tables incur large memory overheads, while Bloom filter–based methods suffer from false positives. To overcome these limitations, the paper proposes the Exact Binary Set Separator (XBSS), an exact query structure built upon a balanced binary tree. XBSS decouples the control plane from the data plane to enable efficient dynamic updates and employs a minimization scheme requiring only two hash computations, thereby guaranteeing 100% accuracy while substantially reducing computational overhead. Experimental results demonstrate that XBSS achieves a query throughput exceeding 120 million operations per second—20% higher than Coloring Embedder and 21.6× faster than Ludo hashing—while maintaining a compact memory footprint.

Abstract

Multi-set membership queries are ubiquitous in networking and database systems. Current solutions force a difficult compromise: hash tables guarantee correctness but suffer from high memory footprints, while filter-based approaches optimize space at the cost of probabilistic errors. In this paper, we propose STEM2, a fast and space-efficient data structure that achieves 100% query accuracy and can support dynamic key updates for multi-set membership queries. STEM2 utilizes a balanced binary tree architecture where each non-leaf node incorporates a novel Exact Binary Set Separator (XBSS) to partition keys into two disjoint groups. A key innovation of our design is a minimized hashing scheme that requires only two hash computations per key lookup, significantly reducing computational overhead. Additionally, STEM2 separates the control plane and the data plane: the control plane handles construction and dynamic updates, while the data plane is dedicated to serving efficient membership queries. Extensive experiments show that STEM2 achieves over 120 million operations per second (Mops) in lookup throughput, outperforming the state-of-the-art Coloring Embedder by 20% and the Ludo hashing by up to 21.6X, while maintaining compact memory cost and exact correctness.

Problem

• multi-set membership query
• space efficiency
• exact correctness
• dynamic updates

Innovation

• Exact Multi-Set Membership
• Balanced Binary Tree
• Exact Binary Set Separator
• Minimized Hashing
• Control-Data Plane Separation