ClockRoPE: Random Fourier Rotations for Temporal Routine Modeling

2026-07-28

AI Summary

This work addresses the limitation of conventional Rotary Position Embedding (RoPE), whose log-linear frequency scheduling struggles to capture complex temporal periodicities in sequential recommendation. The authors propose ClockRoPE, the first position encoding method that incorporates random Fourier feature theory to model user behavioral rhythms through rotation frequencies derived from a periodic attention modulation function. Grounded in the theoretical insight that any normalized continuous positive-definite modulation function can be approximated by random rotations induced by its Fourier transform, ClockRoPE flexibly captures multi-scale periodic dependencies. Online A/B tests within a large-scale generative retrieval system on a major video platform demonstrate that ClockRoPE significantly improves key engagement metrics and has been successfully deployed in production.

Abstract

Rotary Position Embedding (RoPE) has been widely adopted in transformer-based large language models. However, its log-linear frequency schedule, originally designed to produce long-term attention decay, limits its adoption in domains with more complex distance-correlation patterns, such as temporal periodicity in sequential recommendation. We investigate the expressiveness of general query/key rotations and find that any normalized continuous positive-definite attention modulation function can be approximated by random rotations induced by its own Fourier transform, which we term Random Fourier Rotations. Building on this theory, we propose ClockRoPE for routine modeling in sequential recommendation, where rotation frequencies are derived from periodic attention modulation functions. In online A/B tests, ClockRoPE demonstrates consistent improvements in valued engagement metrics, and has been successfully deployed in production-scale generative retrieval system at a major video-sharing platform.

Problem

• Rotary Position Embedding
• temporal periodicity
• sequential recommendation
• attention modulation
• time-aware modeling

Innovation

• Random Fourier Rotations
• Rotary Position Embedding
• Temporal Periodicity
• Sequential Recommendation
• Attention Modulation