Scale-Invariant Learning-to-Rank

2024-10-02

ACM Conference on Recommender Systems

AI Summary

To address the degradation in ranking performance caused by inconsistent feature scaling between training and online inference in Learning-to-Rank (LTR), this paper proposes a mathematically rigorous deep-and-wide joint modeling paradigm that guarantees scale invariance. It is the first LTR method to achieve end-to-end, normalization-free theoretical scale invariance, enabling automatic adaptation to arbitrary feature rescalings throughout both training and inference—thereby eliminating reliance on standardization or batch normalization. The approach integrates co-designed architecture, explicit scale-invariance constraints, and a perturbation-robustness evaluation framework. Under simulated scale-mismatch scenarios, it significantly outperforms baselines: Ranking Loss decreases by 12.7%, NDCG@10 improves by 8.3%, with zero additional inference latency.

Technology Category

• Machine Learning: Learning Preferences or Rankings
• Search and Optimization: Learning to Search
• Natural Language Processing: Learning & Optimization for NLP

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs

Abstract

At Expedia, learning-to-rank (LTR) models plays a key role on our website in sorting and presenting information more relevant to users, such as search filters, property rooms, amenities, and images. A major challenge in deploying these models is ensuring consistent feature scaling between training and production data, as discrepancies can lead to unreliable rankings when deployed. Normalization techniques like feature standardization and batch normalization could address these issues but are impractical in production due to latency impacts and the difficulty of distributed real-time inference. To address consistent feature scaling issue, we introduce a scale-invariant LTR framework which combines a deep and a wide neural network to mathematically guarantee scale-invariance in the model at both training and prediction time. We evaluate our framework in simulated real-world scenarios with injected feature scale issues by perturbing the test set at prediction time, and show that even with inconsistent train-test scaling, using framework achieves better performance than without.

Problem

• Learning to Rank
• Data Inconsistency
• Model Degradation

Innovation

• Deep and Wide Neural Networks
• Learning to Rank (LTR) Framework
• Robustness to Data Variability