UniRank: Benchmarking Ranking Models for Unified Sequential Modeling and Feature Interaction

2026-07-22

AI Summary

Existing unified sequence modeling and ranking approaches are difficult to reproduce and fairly compare due to their reliance on proprietary data and closed-source implementations. This work proposes UniRank, an open-source benchmark that establishes the first reproducible and open evaluation framework specifically designed for unified ranking models. It introduces a temporal pointwise autoregressive supervision scheme and a standardized multi-task evaluation protocol, accompanied by an efficient PyTorch toolkit supporting distributed training, mixed-precision computation, and optimized attention and operator implementations. The framework evaluates 15 representative models across five large-scale public datasets—spanning up to over 700 million samples and sequences as long as 10⁵ interactions—significantly lowering hardware barriers and enabling systematic academic investigation into scaling laws, long-sequence modeling, and multi-task ranking, thereby bridging the gap between industrial practice and academic research.

Abstract

Ranking is a core stage in online advertising and recommender systems. Modern ranking models increasingly unify sequential modeling and feature interaction, yet many advances rely on proprietary data, closed implementations, and large-scale industrial infrastructure. This setting limits reproducible comparison and hinders academic study of scaling laws, long-sequence modeling, and multi-task ranking. To address these limitations, this paper proposes UniRank, an open benchmark for ranking models that unify sequential modeling and feature interaction. UniRank uses chronological pointwise autoregressive supervision, standardizes evaluation across feedback tasks, and provides a PyTorch toolkit with Distributed Data Parallel training, operator optimization, mixed-precision training, attention optimization, and other efficiency techniques that reduce hardware requirements. We benchmark 15 representative unified ranking models on five large-scale public datasets from short-video, advertising, and e-commerce platforms, with the largest dataset containing over 700 million instances and the longest behavior sequence exceeding 10^5 interactions. UniRank provides a reproducible basis for comparing unified ranking models, studying scaling laws under limited compute, and narrowing the gap between academic and industrial ranking research. We believe UniRank benefits researchers, practitioners, and beginners through reproducible experiments, production-oriented evaluation, and accessible implementations. Code and data are available at https://github.com/salmon1802/UniRank.

Problem

• ranking
• sequential modeling
• feature interaction
• reproducibility
• benchmarking

Innovation

• unified ranking
• sequential modeling
• feature interaction
• open benchmark
• efficient training