Auditing Semantic Gains in Sequential Recommendation: A Lightweight Recovery Test

2026-08-02

AI Summary

This work aims to disentangle the true sources of performance gains in semantic models for sequential recommendation—specifically, whether improvements stem from language model reasoning, semantic ID generation, end-to-end training, or the complementary nature of offline semantic representations and collaborative signals. To this end, the authors propose LIME-Rec, a lightweight and auditable multi-expert fusion framework that integrates SASRec, ItemCF, and frozen text embeddings (BAAI/bge-base-en-v1.5). By employing score normalization, an interpretable fusion gating mechanism, and bounded historical calibration, LIME-Rec effectively isolates semantic contributions. Experiments demonstrate that transparent fusion with offline semantic representations alone yields substantial gains, achieving R@10 scores of 0.0996, 0.1105, and 0.0593 on Amazon Beauty, Toys, and Sports, respectively—surpassing the strongest baseline by 7.0%–12.0%. Performance degrades by 13.6%–17.5% under semantic misalignment, underscoring the critical role of semantic coherence.

Abstract

Recent semantic and generative-retrieval recommenders report substantial improvements over ID-only sequential baselines, but it remains unclear whether these gains arise from language-model reasoning, semantic-ID generation, end-to-end semantic architectures, stronger offline item representations, or complementary semantic and collaborative signals. We investigate this attribution ambiguity through LIME-Rec, a lightweight and auditable recovery test. LIME-Rec combines three independent experts: a SASRec sequential expert, an ItemCF co-occurrence expert, and a semantic expert based on frozen BAAI/bge-base-en-v1.5 item embeddings. Their full-catalog scores are normalized per user and combined through auditable score-level fusion followed by bounded history calibration. The fusion gate and calibration head are fitted on validation data only, require no serving-time language-model inference, and keep each expert contribution separately inspectable. On Amazon Beauty, Toys, and Sports, LIME-Rec achieves R@10 scores of 0.0996, 0.1105, and 0.0593, outperforming the strongest comparison baseline by 7.0%-12.0%. Three-expert fusion without history calibration consistently outperforms calibrated SASRec, showing that calibration alone does not explain the recovery. Randomly permuting item-text embeddings across item IDs reduces R@10 by 13.6%-17.5%, indicating that the gains depend on genuine item-text correspondence rather than additional representation capacity. These results suggest that lightweight recovery from offline item representations and transparent fusion should be ruled out before improvements are attributed to serving-time language modeling, semantic-ID generation, or heavier semantic machinery.

Problem

• sequential recommendation
• semantic gains
• attribution ambiguity
• recommendation systems
• language modeling

Innovation

• semantic recommendation
• auditable fusion
• lightweight recovery test
• offline item representations
• attribution analysis