RoleMix: Unifying Sequential and Non-Sequential Features via Semantic Tokenization for Post-Click Conversion Rate Prediction

2026-07-18

AI Summary

This work addresses the structural mismatch between sparse multi-field features and long sequential user behaviors in post-click conversion rate prediction by introducing a semantic tokenization mechanism. This approach explicitly converts non-sequential features into tokens that preserve their semantic roles, enabling early and deep fusion of sequential and non-sequential signals through a unified token interface. The method employs a two-stage hierarchical windowed attention scheme to compress user behavior sequences and incorporates a lightweight UniMixing-Lite module for joint optimization. Evaluated on the KDD Cup 2026 Tencent UniRec Challenge, the proposed model achieves an online AUC of 83.648%, significantly outperforming the official baseline by 1.953%, thereby demonstrating the critical contribution of semantic tokenization to performance improvement.

Abstract

Post-click conversion rate (PCVR) prediction is central to industrial recommendation, but remains challenged by the structural mismatch between sparse, unordered multi-field features and long, domain-specific behavior histories. Existing models often process these signals through separate pathways and fuse them late, weakening semantic roles and limiting cross-signal refinement. We propose RoleMix, a unified interaction architecture that represents sequential and non-sequential evidence through a shared, role-preserving token interface. Non-sequential fields are converted into explicit semantic tokens that preserve user, item, pairwise, dense, contextual, and cross-feature roles, while long behavior domains are compressed into item- and context-aware sequence-query tokens through two-stage hierarchical window attention. The resulting global, semantic, and sequence-query tokens are jointly refined by stacked UniMixing-Lite blocks for PCVR prediction. On the large-scale KDD Cup 2026 Tencent UniRec Challenge, RoleMix achieves 83.648% online AUC, outperforming the official industrial baseline by 1.953%. Ablation studies show that semantic tokenization yields the largest isolated gain, highlighting a key principle for large-scale PCVR modeling: preserving field semantics at the token-interface level is as important as scaling the interaction backbone.

Problem

• post-click conversion rate
• structural mismatch
• multi-field features
• behavior histories
• semantic roles

Innovation

• semantic tokenization
• unified interaction architecture
• hierarchical window attention
• role-preserving tokens
• post-click conversion rate prediction