Engineering Select Support for Hybrid Bitvectors

2025-09-08

AI Summary

Existing hybrid bit vectors support only rank queries, lacking efficient select support—severely limiting their applicability in compressed data structures such as text indexes and graphs. This work presents the first efficient select query support for hybrid bit vectors. We propose an adaptive block-wise encoding scheme that dynamically selects run-length, plain, or sparse encoding based on local redundancy, and extend the index structure to jointly optimize both rank and select operations. Our approach achieves excellent time–space trade-offs across both repetitive and non-repetitive datasets. Experimental results show that query throughput matches that of the fastest existing bit vector implementations, while space overhead approaches that of the most compact alternatives. By unifying efficient rank and select support within a single hybrid framework, this work significantly broadens the practical utility and performance envelope of hybrid bit vectors.

Technology Category

• Data Mining & Knowledge Management: Data Compression
• Search and Optimization: Mixed Discrete/Continuous Search
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• Graph Algorithms and Modeling for the Web: Querying, indexing, and retrieval in Web-related graphs
• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

One of the central problems in the design of compressed data structures is the efficient support for rank and select queries on bitvectors. These two operations form the backbone of more complex data structures (such as wavelet trees) used for the compact representation of texts, trees, graphs, or grids. Their efficient implementation is one of the most frequently studied problems in compressed data structures. One effective solution is the so-called hybrid bitvector implementation, which partitions the input bitvector into blocks and adaptively selects an encoding method, such as run-length, plain, or minority encoding, based on local redundancy. Experiments have shown that hybrid bitvectors achieve excellent all-around performance on repetitive and non-repetitive inputs. However, current implementations support only rank queries (i.e., counting the number of ones up to a given position) and lack support for select queries. This limitation significantly restricts their applicability. In this paper, we propose a method to add support for select queries to hybrid bitvectors, and we conduct an extensive set of experiments. Our results show that hybrid bitvectors offer excellent performance, matching the speed of the fastest and the space efficiency of the most compact existing bitvectors.

Problem

• Adding select query support to hybrid bitvectors
• Overcoming limitations in compressed data structure operations
• Enhancing hybrid bitvector applicability for complex data structures

Innovation

• Hybrid bitvector with adaptive block encoding
• Adding select query support to hybrid bitvectors
• Maintaining speed and space efficiency simultaneously