Factorized Hypothesis Search for Evidence-to-Taxonomy Retrieval

2026-08-06

AI Summary

This work addresses the "retrieval readiness gap"—a performance bottleneck in large-scale classification systems when inputs consist solely of indirect evidence (e.g., table cells) and suffer from semantic ambiguity. To overcome this, the authors propose a factorized hypothesis search mechanism that decomposes semantic interpretation into composable, named dimensional hypotheses. By leveraging structured query generation, parallel multi-hypothesis retrieval, and dimension-level candidate validation, the approach circumvents reliance on free-form text generation. Evaluated on financial taxonomy labeling and the CodiEsp clinical coding task, the method significantly outperforms existing non-oracle approaches, achieving consistent improvements in Recall@1, Mean Reciprocal Rank (MRR), and final accuracy, thereby demonstrating its effectiveness and robustness.

Abstract

Large-taxonomy retrieval often assumes that the input already expresses the target concept. In many settings, however, the input is indirect evidence, such as a table cell whose meaning depends on its row, column, datatype, and context. We call this mismatch the retrieval readiness gap. Our analysis shows that the current index retrieves the target reliably when its semantics are explicit, while raw evidence often leaves it deep in the ranking. We propose Factorized Hypothesis Search (FHS), which maintains multiple partial interpretations over named semantic dimensions. These hypotheses support structured query rendering, multi-hypothesis retrieval, and dimension-level candidate verification. On both financial taxonomy tagging and CodiEsp clinical coding tasks, FHS achieves the best Recall@1, MRR, and final accuracy among the non-oracle methods. Replacing the factorized hypothesis path with a free-text ensemble causes the largest drop in head-ranking performance, while sequential refinement provides no additional gain over FHS's strong parallel first round.

Problem

• large-taxonomy retrieval
• retrieval readiness gap
• indirect evidence
• semantic interpretation
• evidence-to-taxonomy retrieval

Innovation

• Factorized Hypothesis Search
• retrieval readiness gap
• semantic dimensions
• structured query rendering
• multi-hypothesis retrieval