Beyond Relevance: Structured Semantic Supervision for Product Search with LLM-Augmented Annotations

2026-09-20

AI Summary

该研究通过结合大语言模型生成的结构化属性和人工验证的标注，改进电商搜索中商品与查询的相关性判断，使用双编码检索器和MLP重排序器进行评估。

Abstract

E-commerce search requires distinguishing products that are merely related to a query from those that directly satisfy the user's shopping intent. We augment query-product pairs with structured LLM-generated query and product attributes and human-validated relevance, explanations, and centrality judgments, and evaluate these signals using a simple dual-encoder retriever and MLP re-ranker. On an augmented subset of ESCI, a human-feature oracle reaches $0.9382$ nDCG@10, while a human-free trained $Q+P$ configuration reaches $0.9258$. Synthetic approximations of the human signals reach $0.9150$ overall but provide substantial gains for difficult, low-performing queries. Ablations show that most of the oracle improvement comes from post-edited explanations and annotator comments rather than the scalar centrality feature, suggesting that LLMs are most useful for exposing and approximating structured semantic supervision rather than replacing human judgment directly.

Problem

• E-commerce search
• shopping intent
• query-product pairs
• relevance
• structured semantic supervision

Innovation

• structured semantic supervision
• LLM-generated attributes
• human-validated relevance
• explanations and centrality judgments