LeAP: Learnable Adaptive Permutation for Feature Selection in Heterogeneous and Sparse Recommender Systems

2026-05-31

AI Summary

This work addresses the efficiency and accuracy bottlenecks in feature selection for recommender systems caused by heterogeneous, high-dimensional, and extremely sparse features. To this end, we propose LeAP, a learnable adaptive permutation mechanism that transforms conventional random permutations into a trainable process. LeAP incorporates an adaptive regularization strategy tailored to handle dimensional heterogeneity and extreme sparsity, enabling efficient and accurate feature importance estimation. Designed as a model-agnostic, plug-in module, LeAP is compatible with arbitrary recommendation models and scales to ultra-large sparse inputs. Empirical evaluations on four public benchmarks demonstrate state-of-the-art performance. Furthermore, deployment in an industrial search ranking system—processing over one billion daily requests and featuring a 2TB-scale parameter space—successfully eliminates more than 3,600 redundant features, achieving 2–10× speedups over baseline methods.

Abstract

Modern industrial recommender systems rely on thousands of heterogeneous features -- ranging from low-dimensional scalars (e.g., statistical value) to high-dimensional embeddings (e.g., user-id embeddings, MLP representations) -- to achieve high-precision predictions. Given the immense computational costs associated with training, efficient feature selection is critical. However, existing methods encounter three primary bottlenecks: (1) they typically assume uniform feature dimensions or require costly mapping to a fixed size; (2) they struggle with extreme sparsity, where the majority of features (e.g., 99%+) remain at default values; and (3) traditional permutation-based approaches are computationally prohibitive in large-scale settings. To address these challenges, we propose LeAP (Learnable Adaptive Permutation), a novel, model-agnostic plug-in module for feature selection. LeAP transforms the inefficient random permutation process into a learnable mechanism, significantly accelerating the evaluation of feature importance. In addition, we introduce an adaptive regularization strategy tailored for heterogeneous dimensions and extreme sparsity, enabling superior feature importance ranking results across asymmetric input spaces. Experiments on four public recommendation datasets demonstrate that LeAP achieves state-of-the-art performance. Furthermore, LeAP has been deployed in a large-scale industrial search ranking model with over a billion daily requests and a 2TB model parameter scale. In this real-world scenario involving 12,000+ total feature dimensions, LeAP successfully identified and removed over 3,600 redundant dimensions without performance degradation, which is 2 to 10 times the ability of compared baseline methods.

Problem

• feature selection
• heterogeneous features
• sparse recommender systems
• high-dimensional embeddings
• extreme sparsity

Innovation

• Learnable Adaptive Permutation
• Feature Selection
• Heterogeneous Features
• Extreme Sparsity
• Model-Agnostic