AI-University: An LLM-based platform for instructional alignment to scientific classrooms

2025-04-11

AI Summary

To address the misalignment between AI-driven instructional content and instructors’ pedagogical styles in university science classrooms, this paper proposes a teaching-style-aware LLM fine-tuning framework that integrates retrieval-augmented generation (RAG) with LoRA-based efficient adaptation, enabling personalized and traceable course delivery. The framework jointly models heterogeneous multimodal data—including lecture videos, slides, and textbooks—to support precise response attribution to video timestamps and textbook passages. Its key innovations include: (1) the first explicit incorporation of pedagogical style modeling into the LLM fine-tuning pipeline, and (2) a dual-verification mechanism combining automated LLM evaluation with expert educator assessment. In empirical evaluation on a Finite Element Method (FEM) course, the expert-tuned model outperformed baseline models on 86% of test cases; LLM-based adjudication further confirmed its superiority over Llama 3.2 in 80% of comparisons. A fully traceable web-based prototype system has been deployed.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval
• Humans and AI: Human-in-the-loop Machine Learning

Application Category

• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

We introduce AI University (AI-U), a flexible framework for AI-driven course content delivery that adapts to instructors' teaching styles. At its core, AI-U fine-tunes a large language model (LLM) with retrieval-augmented generation (RAG) to generate instructor-aligned responses from lecture videos, notes, and textbooks. Using a graduate-level finite-element-method (FEM) course as a case study, we present a scalable pipeline to systematically construct training data, fine-tune an open-source LLM with Low-Rank Adaptation (LoRA), and optimize its responses through RAG-based synthesis. Our evaluation - combining cosine similarity, LLM-based assessment, and expert review - demonstrates strong alignment with course materials. We also have developed a prototype web application, available at https://my-ai-university.com, that enhances traceability by linking AI-generated responses to specific sections of the relevant course material and time-stamped instances of the open-access video lectures. Our expert model is found to have greater cosine similarity with a reference on 86% of test cases. An LLM judge also found our expert model to outperform the base Llama 3.2 model approximately four times out of five. AI-U offers a scalable approach to AI-assisted education, paving the way for broader adoption in higher education. Here, our framework has been presented in the setting of a class on FEM - a subject that is central to training PhD and Master students in engineering science. However, this setting is a particular instance of a broader context: fine-tuning LLMs to research content in science.

Problem

• Adapts AI-driven course content to instructors' teaching styles
• Generates instructor-aligned responses from lecture materials using LLM
• Enhances traceability of AI-generated responses in educational settings

Innovation

• LLM fine-tuning with LoRA for alignment
• RAG-based synthesis for optimized responses
• Web app linking responses to materials