Technology Category
Search and Optimization: Learning to SearchMachine Learning: Learning Preferences or RankingsKnowledge Representation and Reasoning: Qualitative Reasoning
LORE: A Large Generative Model for Search Relevance
🤖 AI Summary
Existing e-commerce search relevance modeling treats relevance as a monolithic task, lacking systematic decomposition into core capabilities—leading to performance bottlenecks. Method: We propose a qualitatively driven relevance capability decomposition framework that disentangles relevance into three orthogonal, modelable sub-capabilities: knowledge reasoning, multimodal matching, and rule adherence. Leveraging large language models (LLMs), we design a two-stage training paradigm—supervised fine-tuning (SFT) to generate structured chain-of-thought (CoT) rationales, followed by reinforcement learning from human feedback (RLHF) to align with human preferences—and a query-frequency-aware hierarchical deployment strategy, overcoming key limitations of conventional CoT approaches. Contribution/Results: Our method achieves a 27% improvement in the online GoodRate metric and establishes a complete LLM-based relevance operationalization pipeline, spanning data curation, model training, and engineering deployment.
Technology Category
Application Category
📝 Abstract
Achievement. We introduce LORE, a systematic framework for Large Generative Model-based relevance in e-commerce search. Deployed and iterated over three years, LORE achieves a cumulative +27% improvement in online GoodRate metrics. This report shares the valuable experience gained throughout its development lifecycle, spanning data, features, training, evaluation, and deployment. Insight. While existing works apply Chain-of-Thought (CoT) to enhance relevance, they often hit a performance ceiling. We argue this stems from treating relevance as a monolithic task, lacking principled deconstruction. Our key insight is that relevance comprises distinct capabilities: knowledge and reasoning, multi-modal matching, and rule adherence. We contend that a qualitative-driven decomposition is essential for breaking through current performance bottlenecks. Contributions. LORE provides a complete blueprint for the LLM relevance lifecycle. Key contributions include: (1) A two-stage training paradigm combining progressive CoT synthesis via SFT with human preference alignment via RL. (2) A comprehensive benchmark, RAIR, designed to evaluate these core capabilities. (3) A query frequency-stratified deployment strategy that efficiently transfers offline LLM capabilities to the online system. LORE serves as both a practical solution and a methodological reference for other vertical domains.
Problem
Research questions and friction points this paper is trying to address.
Develops a generative model for e-commerce search relevance
Decomposes relevance into knowledge, reasoning, and rule adherence
Provides a training and deployment blueprint for LLM relevance lifecycle
Innovation
Methods, ideas, or system contributions that make the work stand out.
Two-stage training with SFT and RL
RAIR benchmark for core capabilities
Query frequency-stratified deployment strategy
🔎 Similar Papers
No similar papers found.
