CORE: A Unified Cascaded Ordinal Relevance Estimation Framework for E-commerce Search

2026-07-27

AI Summary

This work addresses the mismatch between optimization objectives and evaluation metrics in e-commerce search relevance ranking, which commonly treats relevance as a nominal multi-class classification problem and thereby ignores the inherent ordinal nature of relevance levels. To resolve this, the authors propose a unified cascaded ordinal relevance estimation framework that reformulates multi-class prediction as a sequence of ordered binary classification decisions from high to low relevance tiers. The approach integrates stepwise reasoning and pruning strategies powered by large language models, a hierarchical reward mechanism, BERT-based multi-level binary classification heads, and hierarchical distillation techniques to achieve ordinal-aware relevance modeling. Extensive offline evaluations on industrial-scale data and online A/B tests demonstrate significant performance gains, with a 15.94% reduction in bad-case rate.

Abstract

Ranking relevance is a fundamental task in e-commerce search, directly affecting ranking quality and consumer experience. Although inherently an ordinal classification problem, it is commonly formulated as conventional multi-class classification, which overlooks the natural order among relevance levels and assigns equal penalties to adjacent and distant misclassifications. This mismatch leads to suboptimal learning objectives for practical relevance evaluation. To address this issue, we propose a unified cascaded binary classification framework applicable to both large language model inference and online BERT-based inference, which reformulates relevance estimation as a sequential decision process and decomposes multi-class prediction into a series of ordered binary judgments from higher to lower relevance tiers. For large language models, we design a step-wise reasoning procedure with pruning strategies and tier-specific reward functions. For the online BERT model, we replace the conventional classification head with multiple level-wise binary classifiers and distill the capabilities of large language models into the online model. Extensive offline industrial benchmark evaluations and online A/B experiments demonstrate that the proposed framework substantially improves relevance performance, reducing the online bad-case rate by 15.94\%. Further analyses suggest that tier-wise modeling is effective for relevance estimation.

Problem

• ordinal classification
• relevance estimation
• e-commerce search
• ranking relevance
• multi-class classification

Innovation

• ordinal classification
• cascaded binary classification
• relevance estimation
• knowledge distillation
• e-commerce search