LlamaRec-LKG-RAG: A Single-Pass, Learnable Knowledge Graph-RAG Framework for LLM-Based Ranking

2025-06-09

AI Summary

Existing RAG-based recommender systems predominantly rely on flat similarity search, neglecting the structured relational patterns inherent in user-item interactions. This paper introduces the first end-to-end differentiable, single-forward-pass RAG framework that dynamically constructs personalized knowledge subgraphs and injects them into Llama-2 prompts to enable efficient and interpretable ranking. Key contributions include: (1) a lightweight, knowledge-graph-path-aware user preference module that enables heterogeneous relation fusion within a single inference step; and (2) the first jointly differentiable training of knowledge graph–enhanced RAG and LLM-based recommendation ranking. Evaluated on ML-100K and Amazon Beauty, our method achieves average improvements of 12.7% in MRR, 14.3% in NDCG@10, and 15.1% in Recall@10 over state-of-the-art baselines—including LlamaRec—demonstrating superior effectiveness and interpretability.

Technology Category

Data Mining & Knowledge Management: Recommender SystemsMachine Learning: Learning Preferences or RankingsKnowledge Representation and Reasoning: Preferences

Application Category

User Modeling, Personalization and Recommendation: Fairness-aware retrieval and rankingSearch and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAGGraph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Recent advances in Large Language Models (LLMs) have driven their adoption in recommender systems through Retrieval-Augmented Generation (RAG) frameworks. However, existing RAG approaches predominantly rely on flat, similarity-based retrieval that fails to leverage the rich relational structure inherent in user-item interactions. We introduce LlamaRec-LKG-RAG, a novel single-pass, end-to-end trainable framework that integrates personalized knowledge graph context into LLM-based recommendation ranking. Our approach extends the LlamaRec architecture by incorporating a lightweight user preference module that dynamically identifies salient relation paths within a heterogeneous knowledge graph constructed from user behavior and item metadata. These personalized subgraphs are seamlessly integrated into prompts for a fine-tuned Llama-2 model, enabling efficient and interpretable recommendations through a unified inference step. Comprehensive experiments on ML-100K and Amazon Beauty datasets demonstrate consistent and significant improvements over LlamaRec across key ranking metrics (MRR, NDCG, Recall). LlamaRec-LKG-RAG demonstrates the critical value of structured reasoning in LLM-based recommendations and establishes a foundation for scalable, knowledge-aware personalization in next-generation recommender systems. Code is available at~href{https://github.com/VahidAz/LlamaRec-LKG-RAG}{repository}.

Problem

• Enhancing LLM-based ranking with structured knowledge graphs
• Overcoming flat retrieval limitations in RAG frameworks
• Integrating personalized user-item relations for better recommendations

Innovation

• Integrates personalized knowledge graph into LLM ranking
• Uses lightweight module for dynamic relation path identification
• Unifies inference with fine-tuned Llama-2 for recommendations