🤖 AI Summary
This study replicates and validates the XRec interpretable recommendation framework, focusing on its adaptability and generalization to open-source large language models (LLMs), particularly Llama 3.
Method: We propose an enhanced Mixture of Experts (MoE) embedding architecture that jointly incorporates collaborative signals and language modeling capabilities. Systematic ablation studies are conducted via instruction tuning, collaborative learning, and input/output embedding optimization.
Contribution/Results: (1) First successful replication of XRec’s core performance on Llama 3, outperforming baseline methods on several metrics; (2) identification of expert module embedding structure as critical for explanation stability and personalization; (3) public release of evaluation code and experimental configurations to foster reproducible research in interpretable recommendation. Results show that integrating collaborative information significantly improves explanation consistency, yet overall performance does not uniformly surpass all strong baselines.
📝 Abstract
In this study, we reproduced the work done in the paper "XRec: Large Language Models for Explainable Recommendation" by Ma et al. (2024). The original authors introduced XRec, a model-agnostic collaborative instruction-tuning framework that enables large language models (LLMs) to provide users with comprehensive explanations of generated recommendations. Our objective was to replicate the results of the original paper, albeit using Llama 3 as the LLM for evaluation instead of GPT-3.5-turbo. We built on the source code provided by Ma et al. (2024) to achieve our goal. Our work extends the original paper by modifying the input embeddings or deleting the output embeddings of XRec's Mixture of Experts module. Based on our results, XRec effectively generates personalized explanations and its stability is improved by incorporating collaborative information. However, XRec did not consistently outperform all baseline models in every metric. Our extended analysis further highlights the importance of the Mixture of Experts embeddings in shaping the explanation structures, showcasing how collaborative signals interact with language modeling. Through our work, we provide an open-source evaluation implementation that enhances accessibility for researchers and practitioners alike. Our complete code repository can be found at https://github.com/julianbibo/xrec-reproducibility.