CCFormer: Efficient Cross-Field Interaction and Hierarchical Sequence Compression for Industrial Recommendation at Tencent

2026-07-30

AI Summary

This work addresses the challenge of efficiently modeling long user behavior sequences and capturing fine-grained cross-feature-field interactions under stringent latency and resource constraints in industrial recommendation systems. To this end, we propose CCFormer, an efficient Transformer-based architecture that innovatively integrates feature-field-separated cross-attention with a hierarchical sequence compression mechanism. The former enhances fine-grained interactions across heterogeneous feature fields, while the latter enables low-loss sequence compression through progressively expanded receptive fields. Evaluated on both public benchmarks and Tencent’s large-scale industrial platform, CCFormer significantly outperforms state-of-the-art methods, achieving a 3.57% increase in click-through rate and a 1.71% gain in advertising revenue in online A/B tests, along with a 2.21× acceleration in training speed.

Abstract

Recent studies in industrial recommendation systems have demonstrated that sequential recommendation models built upon self-attention can benefit from predictable scaling laws by increasing sequence length and model capacity. However, practical recommender systems impose strict latency and resource constraints, making it challenging to balance computational overhead with fine-grained feature interaction. In this paper, we propose CCFormer, an efficient Transformer backbone that unifies cross-field feature interaction and compressed long-sequence modeling for industrial recommendation. Specifically, CCFormer combines feature-field separated cross attention with long-sequence subspace token mixing to exploit long-term preference signals across heterogeneous feature domains. A hierarchical sequence compression strategy with progressively expanded receptive fields enables efficient long-sequence modeling with reduced information loss. Extensive experiments on two public benchmarks and a large-scale industrial dataset demonstrate that CCFormer consistently outperforms state-of-the-art baselines. Online A/B tests in a video recommendation scenario and an advertising ranking scenario at Tencent further validate its industrial practicality, yielding a 3.57% CTR gain and a 1.71% advertising revenue lift, respectively, while accelerating model training by 2.21x over the strong HSTU baseline. CCFormer has been fully deployed in Tencent's production recommendation system, serving the main traffic of both scenarios.

Problem

• industrial recommendation
• long-sequence modeling
• cross-field interaction
• computational efficiency
• feature interaction

Innovation

• Cross-field Interaction
• Hierarchical Sequence Compression
• Long-sequence Modeling
• Industrial Recommendation
• Efficient Transformer