Towards Explaining Query Expansion Performance in Information Retrieval

2026-10-07

AI Summary

This study addresses the performance instability of query expansion in information retrieval and the difficulty of any single method achieving universal effectiveness. To this end, it introduces the concept of an "ideal expanded query" alongside a separability measurement framework based on Cohen's d. By employing the BM25 model to quantify score disparities between relevant and non-relevant documents, the research analyzes the underlying mechanisms of performance fluctuations from complementary perspectives, with extensive validation conducted on TREC datasets. The results demonstrate that expansion terms approximating the ideal expanded query significantly enhance retrieval performance. These findings confirm that the proposed separability metric offers an effective theoretical perspective for explaining performance variations in query expansion.

Abstract

Query Expansion (QE) techniques have long been widely used in Information Retrieval (IR) to address the vocabulary mismatch problem. They remain relevant in modern retrieval systems, including those based on large language models (LLMs). However, no single QE method consistently outperforms others across all queries. This work seeks to explain the variation in QE performance through two complementary perspectives. The first is the concept of an Ideal Expanded Query (IEQ)--a hypothetical query that maximizes retrieval effectiveness with a downstream BM25 retrieval model. The second is a separability perspective, which quantifies how distinctly relevant and non-relevant documents are scored for a given expanded query using Cohen's (d). We develop a separability measure and practical formulations to approximate the IEQ and investigate how these factors relate to retrieval effectiveness. Extensive experiments on the TREC Robust collection, TREC DL 2019-2022 passage collections, and TREC DL 2019-2020 document collections reveal several interesting patterns. In particular, we find that expanded queries that are closer to the ideal expanded query tend to achieve higher retrieval effectiveness. We further show that the separability of relevant and non-relevant documents provides a complementary perspective for understanding QE performance.

Problem

• Query Expansion
• Information Retrieval
• Performance Variation
• Ideal Expanded Query
• Separability

Innovation

• Query Expansion
• Ideal Expanded Query
• Separability Measure
• Information Retrieval
• Cohen's d