A Training-Free Large Reasoning Model-based Knowledge Tracing Framework for Unified Prediction and Prescription

2026-01-05

arXiv.org

AI Summary

This work proposes Thinking-KT, a novel framework that achieves unified knowledge tracing without any model training, addressing the instability and high system complexity associated with existing approaches that rely on fine-tuning large language models (LLMs). By leveraging test-time scaling (TTS), zero-shot reasoning, and a unified output structure, Thinking-KT enables a small LLM to simultaneously perform knowledge state prediction, generate personalized feedback, and provide learning recommendations—all without fine-tuning. The method maintains competitive prediction accuracy while significantly reducing system complexity and computational overhead, achieving performance on par with fine-tuned models. These results underscore the critical role of test-time reasoning in advancing intelligent tutoring systems.

Technology Category

• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval
• Machine Learning: Large Multimodal Models (LMMs)
• Knowledge Representation and Reasoning: Knowledge Acquisition

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Knowledge Tracing (KT) aims to estimate a learner's evolving mastery based on interaction histories. Recent studies have explored Large Language Models (LLMs) for KT via autoregressive nature, but such approaches typically require fine-tuning and exhibit unstable or near-random performance. Moreover, prior KT systems primarily focus on prediction and rely on multi-stage pipelines for feedback and recommendation, resulting in increased system complexity and resources. To address this gap, we propose Thinking-KT, a training-free KT framework that incorporates Test-Time Scaling (TTS), enabling even small LLMs to achieve competitive KT performance. Moreover, in this framework, a small LLM can jointly perform KT prediction, personalized feedback generation, and learning recommendation in a unified output without degrading prediction accuracy. Beyond performance, we present the systematic analysis of reasoning traces in KT. Our results demonstrate that TTS is a critical yet underexplored factor in LLM-based KT, and that small LLMs can serve as unified ITS engines.

Problem

• Knowledge Tracing
• Large Language Models
• Training-Free
• Unified Prediction and Prescription
• Test-Time Scaling

Innovation

• Training-Free
• Test-Time Scaling
• Knowledge Tracing
• Unified Prediction and Prescription
• Small LLMs