T-RoPE: Time-Aware Rotary Position Embedding for Sequential Recommendation

2026-09-24

AI Summary

This study addresses the limitation of Rotary Position Embedding (RoPE) in recommender systems, which encodes only sequential order while neglecting temporal periodicity and calendar phases. To overcome this, we propose a time-aware rotary position encoding method. Specifically, it employs a multi-scale frequency bank and non-stationary keys to break temporal translation invariance, and introduces learnable temporal coefficients for shifted query alignment, enabling non-stationary temporal modeling while preserving interface compatibility. Furthermore, timestamp-based angular rotation and a linear-complexity algorithm are designed. Experiments demonstrate that the proposed approach significantly improves performance across multiple benchmark and industrial datasets, achieving a 0.33% increase in conversion rate in online A/B testing.

Abstract

Large-scale recommenders increasingly adopt the sequential generative recipe behind large language models, bringing the Transformer into recommendation along with design choices made for text, including Rotary Position Embedding (RoPE). In language models, RoPE encodes token indices for relative position reasoning, but in recommendation, an interaction index records only event order, saying nothing about elapsed time, behavioral cycles across scales, or calendar phase. We revisit this choice and propose T-RoPE, a time-aware RoPE for sequential generative recommendation that replaces index-only rotation with timestamp-based angles, learnable temporal coefficients, multiscale frequency banks, shifted query alignment, and non-stationary key rotation. We prove that standard RoPE, even on timestamps, remains time-translation invariant and cannot distinguish seasonal contexts, and that T-RoPE breaks this invariance while preserving the RoPE interface. Across five public benchmarks, T-RoPE achieves the best result on every metric on every dataset, improving over the strongest baseline by 78--130\% in HR@10 on the sparse PixelRec data and 8--12\% across metrics on Amazon Books. On an industrial-scale e-commerce dataset with more than 6B interactions, it improves every metric over the HSTU + Time RAB backbone by 13--82\%, with ablations attributing the largest gains to multiscale frequencies ($+56\%$ NDCG@50) and non-stationary keys ($+4\%$). An online A/B test in the Shop app yields positive lifts in conversion rate ($+0.33\%$) and order count ($+0.63\%$). We also provide forward and backward algorithms whose added cost is linear in sequence length and head dimension, keeping time-aware RoPE practical for large generative recommenders.

Problem

• Sequential Recommendation
• Rotary Position Embedding
• Temporal Awareness
• Time-Translation Invariance
• Generative Recommender

Innovation

• Time-Aware Rotary Position Embedding
• Sequential Recommendation
• Multiscale Frequency Banks
• Non-stationary Key Rotation
• Generative Recommender