🤖 AI Summary
Recommender systems suffer from severe long-tail bias due to the power-law distribution of item popularity, leading to over-recommendation of popular items and under-exposure of niche yet high-quality ones. To address this, we propose a model-agnostic, cluster-level multi-objective optimization framework. First, we model the popularity mechanism via causal inference to disentangle confounding effects. Second, we design disparity-driven bipartite clustering to partition items into head and tail clusters, enabling differentiated optimization. Third, we introduce a Pareto-efficient gradient coordination algorithm that jointly optimizes accuracy and fairness objectives, while counterfactual inference mitigates global popularity-induced bias. Evaluated on multiple public benchmarks, our method significantly improves tail-item exposure and NDCG, reduces popularity bias by 32.7%, and preserves head-item recommendation accuracy.
📝 Abstract
Recommender system based on historical user-item interactions is of vital importance for web-based services. However, the observed data used to train the recommender model suffers from severe bias issues. Practically, the item frequency distribution of the dataset is a highly skewed power-law distribution. Interactions of a small fraction of head items account for almost the whole training data. The normal training paradigm from such biased data tends to repetitively generate recommendations from the head items, which further exacerbates the biases and affects the exploration of potentially interesting items from the niche set. In this work, we innovatively explore the central theme of recommendation debiasing from an item cluster-wise multi-objective optimization perspective. Aiming to balance the learning on various item clusters that differ in popularity during the training process, we propose a model-agnostic framework namely Item Cluster-Wise Pareto-Efficient Recommendation (ICPE). In detail, we define our item cluster-wise optimization target as the recommender model should balance all item clusters that differ in popularity, thus we set the model learning on each item cluster as a unique optimization objective. To achieve this goal, we first explore items' popularity levels from a novel causal reasoning perspective. Then, we devise popularity discrepancy-based bisecting clustering to separate the item clusters. Next, we adaptively find the overall harmonious gradient direction for cluster-wise optimization objectives from a Pareto-efficient solver. Finally, in the prediction stage, we perform counterfactual inference to further eliminate the impact of global propensity. Extensive experimental results verify the superiorities of ICPE on overall recommendation performance and biases elimination.
