Knowledge-Geometry Decoupling: Refreshable Pretrained Transfer for Streaming Recommendation

2026-08-03

AI Summary

This work addresses the challenge of transferring and continually updating pre-trained knowledge in recommender systems under behavioral distribution shifts. To this end, the authors propose a knowledge–geometry disentanglement framework that extracts transferable knowledge via Behavior Multi-Token Prediction (BMTP) and decouples knowledge encoding from task-specific geometric learning through read-only cross-attention, Anchored Calibration Residuals (ACR), and orthogonal embedding spaces. This design enables interference-free knowledge updates and efficient downstream adaptation. The method consistently outperforms strong baselines by 4–12% across eight public benchmarks and demonstrates sustained effectiveness on Shopee’s 90-day online streaming data, with A/B tests showing a 1.75% increase in per-user GMV and a 1.53% boost in ad revenue.

Abstract

Industrial recommenders increasingly adopt the pretrain-then-transfer paradigm, yet behavioral distribution drift raises two questions: what to learn from behavior sequences, and how to transfer the learned knowledge while the pretrained model is continually refreshed. To resolve them, we propose Knowledge-Geometry Decoupling (KGD). For what to learn, conventional next-token prediction treats adjacency as dependency and may encode spurious transitions across unrelated sessions. We introduce Behavioral Multi-Token Prediction (BMTP) to retain only collaboratively or semantically related future items as supervision, yielding cleaner and more transferable behavioral knowledge. For how to transfer, pretrained knowledge and task-specific geometry impose conflicting optimization demands on shared parameters. To handle it, KGD assigns them to separate parameter sets: a refreshable encoder owns behavioral knowledge, while a task learner reads contextualized encoder states through read-only cross-attention and writes task-specific geometry through Anchored Calibration Residual (ACR) orthogonal to the pretrained embedding. The decoupled ownership enables continual knowledge refresh without task-gradient interference or invalidating downstream adaptation. KGD improves over strong pretrain-transfer baselines by 4-12% on eight public benchmarks and sustains its advantage over a 90-day production stream where baselines show no gains. KGD has been fully deployed in Shopee. In a live A/B test on Shopee Homepage Search, it increases GMV per user by 1.75% and advertising revenue by 1.53%, demonstrating its high practical value. We provide the core implementation of KGD at https://github.com/FuCongResearchSquad/KGD4REC.

Problem

• behavioral distribution drift
• pretrain-then-transfer
• streaming recommendation
• knowledge transfer
• continual learning

Innovation

• Knowledge-Geometry Decoupling
• Behavioral Multi-Token Prediction
• Anchored Calibration Residual
• Pretrain-then-Transfer
• Streaming Recommendation