WHALE: A Scalable Unified Model for Recommendation with Wukong-HSTU Architecture

2026-07-18

AI Summary

Existing recommender systems struggle to efficiently and cohesively model non-sequential features alongside long user behavior sequences, limiting both model expressiveness and scalability. This work proposes WHALE, a novel architecture that for the first time deeply integrates two scalable backbone networks—Wukong and HSTU. The Wukong module captures high-order interactions among non-sequential features, while the HSTU module efficiently models lengthy behavioral sequences. These components are progressively fused across layers through an attention mechanism, enabling rich cross-modal information exchange. Coupled with custom Triton kernels and a co-designed model-system optimization strategy, WHALE supports end-to-end joint training, achieving significant gains in offline metrics on large-scale industrial datasets and delivering positive online performance improvements with only marginal throughput overhead. The system has been successfully deployed in production.

Abstract

As scalability becomes increasingly important in recommendation modeling, recent architectures have advanced the modeling of two broad sources of ranking signals along separate paths: non-sequence features, including user, item, context, and cross features; and sequence features from user behavior histories. Wukong and HSTU have emerged as representative scalable backbones for these paths: Wukong scales high-order non-sequence feature-interaction modeling, while HSTU scales long user-behavior sequence modeling. Despite their complementary strengths, practical architectures that combine these two types of feature modeling remain underexplored. We present WHALE, a scalable unified recommendation architecture that jointly models non-sequence and sequence features on top of Wukong and HSTU. Each WHALE layer contains a Wukong module, an HSTU module, and an attention-based fusion module in which Wukong-derived interaction representations query HSTU-derived behavior representations. This design keeps both backbones active throughout the network and enables progressive Wukong-HSTU exchange, allowing high-order feature crosses to repeatedly retrieve fine-grained evidence from long user histories. To make WHALE practical for industrial deployment, we introduce customized Triton kernels and other model-systems co-design techniques to improve training and inference efficiency. On large-scale industrial recommendation data, WHALE achieves consistent gains in offline experiments. Additionally, it delivers positive online gains with a modest serving-throughput trade-off. The method has been deployed in production systems. Overall, WHALE provides a practical example of how these two sources of information can be scalably unified in an industrial recommendation model.

Problem

• recommendation
• scalability
• sequence modeling
• non-sequence features
• feature interaction

Innovation

• WHALE
• Wukong-HSTU architecture
• scalable recommendation
• feature-sequence fusion
• model-system co-design