PRECTR: A Synergistic Framework for Integrating Personalized Search Relevance Matching and CTR Prediction

2025-03-24

AI Summary

Existing CTR prediction and search relevance modeling in search-and-recommendation systems suffer from fragmentation, inconsistency, and insufficient personalization. To address these issues, this paper proposes a unified joint modeling framework: (1) a conditional probability fusion mechanism to jointly optimize CTR estimation and relevance scoring; (2) a two-stage training strategy to mitigate convergence difficulties caused by binary CTR loss and exposure bias favoring highly visible but low-relevance items; (3) semantic consistency regularization to align representation spaces; and (4) cross-user similar-query transfer to model personalized relevance preferences. This work presents the first end-to-end joint optimization of deep matching and CTR prediction. Extensive evaluations on production data and online A/B tests demonstrate significant improvements: +9.2% CTR, higher NDCG, improved relevance satisfaction, and an 18.7% reduction in relevance misrecommendation rate.

Technology Category

• Search and Optimization: Learning to Search
• Machine Learning: Learning Preferences or Rankings
• Constraint Satisfaction and Optimization: Search

Application Category

• User Modeling, Personalization and Recommendation: ML for personalized search and recommendations
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Applications of semantic technologies for improving search, browsing, recommendation, personalization

Abstract

The two primary tasks in the search recommendation system are search relevance matching and click-through rate (CTR) prediction -- the former focuses on seeking relevant items for user queries whereas the latter forecasts which item may better match user interest. Prior research typically develops two models to predict the CTR and search relevance separately, then ranking candidate items based on the fusion of the two outputs. However, such a divide-and-conquer paradigm creates the inconsistency between different models. Meanwhile, the search relevance model mainly concentrates on the degree of objective text matching while neglecting personalized differences among different users, leading to restricted model performance. To tackle these issues, we propose a unified extbf{P}ersonalized Search RElevance Matching and CTR Prediction Fusion Model(PRECTR). Specifically, based on the conditional probability fusion mechanism, PRECTR integrates the CTR prediction and search relevance matching into one framework to enhance the interaction and consistency of the two modules. However, directly optimizing CTR binary classification loss may bring challenges to the fusion model's convergence and indefinitely promote the exposure of items with high CTR, regardless of their search relevance. Hence, we further introduce two-stage training and semantic consistency regularization to accelerate the model's convergence and restrain the recommendation of irrelevant items. Finally, acknowledging that different users may have varied relevance preferences, we assessed current users' relevance preferences by analyzing past users' preferences for similar queries and tailored incentives for different candidate items accordingly. Extensive experimental results on our production dataset and online A/B testing demonstrate the effectiveness and superiority of our proposed PRECTR method.

Problem

• Integrates search relevance matching and CTR prediction into one framework
• Addresses inconsistency and lack of personalization in separate models
• Optimizes convergence and prevents irrelevant item recommendations

Innovation

• Unified framework integrates relevance and CTR prediction
• Two-stage training with semantic consistency regularization
• Personalized incentives based on user relevance preferences