BAHSD: Bridging the Long-tail Gap via Adaptive Distillation in Black-box Sequential Recommendation

2026-06-01

AI Summary

This work addresses the challenge of signal heterogeneity in black-box sequential recommendation systems under long-tailed user distributions, where head users suffer from preference固化-induced distillation bias and tail users exhibit sparse, noisy signals. To this end, we propose the first adaptive distillation framework tailored for black-box sequential recommendation. Our approach employs a multi-scale consistency probing mechanism to implicitly assess signal reliability and introduces a hierarchical adaptive objective: for high-confidence signals, it mitigates preference固化 via dynamic-temperature KL divergence; for low-confidence signals, it enhances robustness by integrating ranking consistency with InfoNCE-based contrastive learning. This enables differentiated knowledge transfer across head and tail users, significantly outperforming baselines—improving tail-user performance by over 80% and achieving up to a 4.98% gain over the teacher model—while offering a plug-and-play, high-fidelity black-box extraction solution.

Abstract

Sequential recommendation systems are widely adopted but often deployed as black-box APIs, which has driven recent interest in model extraction to replicate their capabilities locally. However, the long-tail distribution induces severe signal heterogeneity: dense head sequences trigger the solidification of teacher preference, biasing extraction toward local patterns, while sparse tail sequences yield flat, noisy predictions. Existing one-size-fits-all extraction overlooks this disparity, resulting in noise overfitting and suboptimal knowledge transfer. We propose BAHSD, a black-box adaptive distillation framework that handles signal heterogeneity via a multi-scale consistency probing mechanism to implicitly quantify signal reliability. Based on this, an adaptive hierarchical objective is designed: dynamic-temperature KL divergence mitigates preference solidification for high-confidence signals, while ranking consistency and InfoNCE contrastive learning provide noise-robust enhancement for low-confidence signals. BAHSD consistently outperforms baselines, achieving up to 4.98\% gain over the teacher and 80\%+ improvement on tail users, offering a plug-and-play solution for high-fidelity black-box recommendation extraction.

Problem

• long-tail distribution
• black-box sequential recommendation
• model extraction
• signal heterogeneity
• knowledge distillation

Innovation

• adaptive distillation
• black-box sequential recommendation
• long-tail distribution
• signal heterogeneity
• contrastive learning