Robust Fusion of Semantic and Behavioural Signals for LLM Reranking in Personalised Search

2026-09-22

AI Summary

研究解决了个性化搜索中依赖历史行为信号导致的泛化能力下降问题，通过确定性双样本特征dropout训练方法提高系统在有无行为特征时的鲁棒性和性能。

Abstract

Personalised search must satisfy query intent while incorporating user context and historical interactions. LLM-based cross-encoders provide a single reranking interface, but injecting predictive behavioural statistics into their prompts can encourage shortcut learning: reliance on historical signals at the expense of semantic and user-context patterns that generalise to sparse or unseen searches. We study this problem in the personalised search system of a large-scale audio streaming platform using Query Slice Stats (QSS), an interaction-derived behavioural feature summarising historical success for query-candidate pairs. Naive QSS injection improves ranking when the feature is available but reduces robustness when it is removed. We address this with deterministic dual-sample feature-dropout training, which presents each example once with QSS included and once with QSS removed. Offline, QSS injection improves ranking quality by 13.3% when available. Dual-sample training preserves these gains while improving performance under QSS-removed evaluation by 4.0% relative to naive QSS training. In a live online test, both QSS-aware variants improve search success by roughly 2%. The aggregate test does not distinguish dual-sample from features-only training; the cold-start comparison is directionally consistent with the offline results. Paired feature-present and feature-removed training can therefore reduce the tension between exploiting strong behavioural statistics and remaining robust when they are unavailable.

Problem

• personalised search
• LLM reranking
• behavioural signals
• shortcut learning
• historical interactions

Innovation

• deterministic dual-sample feature-dropout training
• Query Slice Stats (QSS)
• personalised search
• robustness