TutorLLM: Customizing Learning Recommendations with Knowledge Tracing and Retrieval-Augmented Generation

2025-01-20

arXiv.org

AI Summary

To address low content relevance and insufficient personalization of large language models (LLMs) in educational applications, this paper proposes a personalized learning recommendation system integrating knowledge tracing (KT) and retrieval-augmented generation (RAG). The method introduces a novel deep coupling between a multi-feature latent-relation BERT-based KT model (MLFBK) and RAG, enabling dynamic modeling of students’ cognitive states and context-aware retrieval of instructional content. Additionally, a web-crawling enhancement module (Scraper) is incorporated to broaden coverage and improve timeliness of educational resources. Experimental results demonstrate a 10% increase in user satisfaction and a 5% improvement in assessment scores over general-purpose LLM baselines. Moreover, the system yields interpretable recommendations, enhancing transparency and pedagogical trustworthiness. This work advances adaptive educational AI by unifying fine-grained cognitive modeling with grounded, up-to-date knowledge retrieval—bridging the gap between theoretical KT frameworks and practical LLM deployment in learning environments.

Technology Category

• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval
• Machine Learning: Large Multimodal Models (LMMs)
• Cognitive Modeling & Cognitive Systems: Adaptive Behavior

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

The integration of AI in education offers significant potential to enhance learning efficiency. Large Language Models (LLMs), such as ChatGPT, Gemini, and Llama, allow students to query a wide range of topics, providing unprecedented flexibility. However, LLMs face challenges, such as handling varying content relevance and lack of personalization. To address these challenges, we propose TutorLLM, a personalized learning recommender LLM system based on Knowledge Tracing (KT) and Retrieval-Augmented Generation (RAG). The novelty of TutorLLM lies in its unique combination of KT and RAG techniques with LLMs, which enables dynamic retrieval of context-specific knowledge and provides personalized learning recommendations based on the student's personal learning state. Specifically, this integration allows TutorLLM to tailor responses based on individual learning states predicted by the Multi-Features with Latent Relations BERT-based KT (MLFBK) model and to enhance response accuracy with a Scraper model. The evaluation includes user assessment questionnaires and performance metrics, demonstrating a 10% improvement in user satisfaction and a 5% increase in quiz scores compared to using general LLMs alone.

Problem

• Enhancing learning efficiency with AI in education
• Addressing content relevance and personalization in LLMs
• Providing dynamic, personalized learning recommendations using KT and RAG

Innovation

• Combines Knowledge Tracing with Retrieval-Augmented Generation
• Uses MLFBK model for personalized learning states
• Enhances accuracy with a Scraper model