RankEvolve: Automating the Discovery of Retrieval Algorithms via LLM-Driven Evolution

2026-02-18

AI Summary

This work proposes a large language model (LLM)-driven program evolution framework that automatically discovers improved lexical retrieval functions by representing candidate algorithms as executable code. Departing from traditional methods like BM25, which rely on manual parameter tuning and struggle to identify superior ranking functions, the approach integrates LLMs with evolutionary search. Starting from seed algorithms such as BM25, it iteratively applies mutation and recombination operators guided by performance feedback across multiple datasets. The evolved retrieval functions consistently outperform strong baselines on BEIR, BRIGHT, and TREC Deep Learning 2019/2020 benchmarks, demonstrating not only significant gains in effectiveness but also strong generalization and cross-dataset transfer capabilities.

Technology Category

• Search and Optimization: Evolutionary Computation
• Machine Learning: Evolutionary Learning
• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval

Application Category

• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Retrieval algorithms like BM25 and query likelihood with Dirichlet smoothing remain strong and efficient first-stage rankers, yet improvements have mostly relied on parameter tuning and human intuition. We investigate whether a large language model, guided by an evaluator and evolutionary search, can automatically discover improved lexical retrieval algorithms. We introduce RankEvolve, a program evolution setup based on AlphaEvolve, in which candidate ranking algorithms are represented as executable code and iteratively mutated, recombined, and selected based on retrieval performance across 12 IR datasets from BEIR and BRIGHT. RankEvolve starts from two seed programs: BM25 and query likelihood with Dirichlet smoothing. The evolved algorithms are novel, effective, and show promising transfer to the full BEIR and BRIGHT benchmarks as well as TREC DL 19 and 20. Our results suggest that evaluator-guided LLM program evolution is a practical path towards automatic discovery of novel ranking algorithms.

Problem

• retrieval algorithms
• automatic discovery
• ranking algorithms
• program evolution
• information retrieval

Innovation

• LLM-driven evolution
• program synthesis
• lexical retrieval
• algorithm discovery
• evolutionary search