SinkRec: Mitigating Semantic State Sink in Long Sequence Recommendation with Memory-Conditioned Gated Delta Networks

2026-06-03

AI Summary

In long-sequence recommendation, linear attention mechanisms are prone to being dominated by repetitive user behaviors due to their compressed recurrent state representation, leading to a phenomenon termed “semantic state sink” that degrades the accuracy of subsequent state retrieval. This work proposes SinkRec, the first framework to formally characterize this issue and address it through a learnable conditional memory mechanism that decouples the storage of collaborative behavior patterns from their dynamic transition modeling. Specifically, repetitive local patterns are externalized as residual quantized codes and reinjected into the system, while a temporal-aware differential gating mechanism, DeltaNet, purifies state read and write operations. Integrating memory-augmented attention with linear computational complexity, SinkRec achieves significant improvements in both recommendation performance and efficiency across multiple public and industrial datasets.

Abstract

Linear attention provides an efficient backbone for long-sequence recommendation by avoiding the quadratic cost of standard Transformers, but its compressed recurrent state can be dominated by repetitive behavior patterns. We identify this phenomenon as semantic state sink, where recurring semantics over-occupy the recurrent state and bias subsequent readouts. To mitigate semantic state sink, we propose SinkRec, a hybrid memory-transition looped architecture that decouples collaborative behavioral pattern storage from dynamic transition modeling. SinkRec externalizes recurring local patterns into a learnable conditional memory through residual vector quantization, reinjects the retrieved codes, and exposes memory key-value pairs to the attention block. It further introduces Temporal-Aware State-Relation Differential Gated DeltaNet (TDGD), which uses memory to purify recurrent writing and reading by suppressing memory-covered updates and removing memory-aligned readout responses. This design turns recurring semantics from state-competing signals into memory-retrievable patterns, allowing the recurrent state to focus on dynamic transitions and alleviating semantic state sink with linear-time efficiency. Experiments on public and industrial datasets demonstrate the effectiveness and efficiency of SinkRec.

Problem

• semantic state sink
• long sequence recommendation
• linear attention
• repetitive behavior patterns
• recurrent state

Innovation

• semantic state sink
• memory-conditioned architecture
• linear attention
• gated DeltaNet
• long-sequence recommendation