LLM-Guided ANN Index Optimization for Human-Object Interaction Retrieval

2026-06-03

AI Summary

This work addresses the challenge of jointly optimizing highly coupled hyperparameters in multi-stage retrieval systems, a task where traditional methods fall short due to their reliance on independence assumptions. The authors propose a stage-aware large language model (LLM) agent that dynamically generates configurations by leveraging the full optimization history, enabling coordinated tuning of approximate nearest neighbor (ANN) index parameters across exploration, exploitation, and fine-tuning phases. To the best of our knowledge, this is the first study to integrate LLMs into coupled hyperparameter optimization for vector retrieval systems, offering not only superior performance but also cross-platform transferability. Experiments demonstrate that the proposed method outperforms Optuna TPE and VDTuner by 33.3% and 34.2%, respectively, on HICO-DET, achieves 15.3× higher throughput than UniIR, and exhibits strong effectiveness and generalization across multiple platforms and datasets.

Abstract

Retrieval systems underpin modern AI applications -- spanning visual search, recommendation engines, and multi-modal question answering. Modern multi-stage retrieval systems require the joint optimization of highly coupled parameters, yet traditional hyperparameter optimization (HPO) methods -- including Tree-structured Parzen Estimators (TPE) and Gaussian Process Bayesian Optimization -- rely on an independence assumption that fundamentally prevents them from navigating these coupled configuration spaces. We address this limitation with a phase-aware large language model (LLM) agent that conditions each proposal on its full optimization history, navigating the coupled parameter space across phase-partitioned exploration, exploitation, and fine-tuning stages. Evaluated on the HICO-DET human-object interaction retrieval benchmark using Intel VDMS (Visual Data Management System), our agent outperforms Optuna TPE by +33.3% and VDTuner by +34.2% under SIEVE (Safeguarded Index Evaluation of Vector-search Efficiency, a quality-constrained throughput metric), delivering a 15.3x throughput gain over UniIR. Validation across three benchmarks confirms that the agent's advantage grows with the degree of parameter coupling: +33.3% on HICO-DET (high coupling), methods converge within 1% on GLDv2 (moderate coupling) and within 3.6% on SIFT1M (near-independent control). Cross-system validation on Milvus confirms the optimizer ranks first on all three datasets without modification, demonstrating transferability across vector database management system (VDBMS) platforms.

Problem

• hyperparameter optimization
• parameter coupling
• multi-stage retrieval
• vector database
• human-object interaction

Innovation

• LLM-guided optimization
• coupled hyperparameter optimization
• phase-aware ANN indexing
• human-object interaction retrieval
• vector database management system