Cascaded Learned Bloom Filter for Optimal Model-Filter Size Balance and Fast Rejection

2025-02-06

AI Summary

Existing learned Bloom filters (LBFs) suffer from imbalanced model size and Bloom structure dimensions, as well as high rejection latency. This paper proposes the Cascaded Learned Bloom Filter (CLBF), the first LBF framework to jointly optimize model size and Bloom component dimensions via dynamic programming—achieving globally optimal trade-offs between them while preserving accuracy. CLBF introduces a multi-level cascaded architecture that enables fine-grained, predictive-hashing co-decision for query routing. Evaluated on real-world datasets, CLBF reduces memory footprint by up to 24% and accelerates rejection latency by up to 14× compared to state-of-the-art methods, significantly improving throughput and resource efficiency.

Technology Category

• Search and Optimization: Learning to Search
• Machine Learning: Learning Preferences or Rankings
• Natural Language Processing: Learning & Optimization for NLP

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

Recent studies have demonstrated that learned Bloom filters, which combine machine learning with the classical Bloom filter, can achieve superior memory efficiency. However, existing learned Bloom filters face two critical unresolved challenges: the balance between the machine learning model size and the Bloom filter size is not optimal, and the reject time cannot be minimized effectively. We propose the Cascaded Learned Bloom Filter (CLBF) to address these issues. Our dynamic programming-based optimization automatically selects configurations that achieve an optimal balance between the model and filter sizes while minimizing reject time. Experiments on real-world datasets show that CLBF reduces memory usage by up to 24% and decreases reject time by up to 14 times compared to state-of-the-art learned Bloom filters.

Problem

• Optimizes model-filter size balance
• Minimizes rejection time effectively
• Enhances memory efficiency significantly

Innovation

• Dynamic programming optimizes model-filter balance
• Cascaded structure minimizes memory usage significantly
• CLBF reduces reject time up to 14 times