VidTutorAssistant: Automating Responses to Programming Tutorial Questions

2026-09-24

AI Summary

This study addresses the challenge of surging viewer questions in programming tutorial video comment sections, which creators struggle to address promptly. To this end, we propose an automated question-answering system based on Retrieval-Augmented Generation (RAG). By integrating the GPT-4 large language model with semantic embeddings and cosine similarity retrieval, the system achieves precise parsing and automated resolution of audience inquiries. Experimental results demonstrate that the system attains a language identification accuracy of 0.99 and a question classification precision of 0.96. Notably, its response correctness (98%) and completeness (99.5%) surpass those of human creators. This work provides an efficient, intelligent solution for facilitating large-scale interactions in online instructional contexts.

Abstract

Programming tutorial videos on YouTube are an important information resource for software developers and students, and their comment sections have evolved into active spaces where viewers ask follow-up questions. The volume of these questions, however, often exceeds what content creators can address, leaving learners without the clarifications they need. We present VidTutorAssistant, a web platform that automates responses to viewer questions on programming video tutorials. VidTutorAssistant implements a retrieval-augmented generation pipeline that extracts a video's transcript, then segments it and embeds it. It then classifies each viewer comment as being a question or non-question, retrieves the most relevant transcript segments to each identified question via cosine similarity, and then generates an answer to the question using an LLM (GPT-4), while grounding the response using the retrieved transcript segments as context. We validate VidTutorAssistant through a study on a subset of 440 user comments selected from a larger dataset of 105,553 comments extracted from 7,522 Python and Java tutorials. VidTutorAssistant is evaluated on various criteria: a) its ability to identify the programming language in a video, achieving a 0.99 accuracy; b) its ability to classify comments into questions and non-questions, reaching a 0.96 accuracy; and c) its ability to produce correct and complete answers to questions, producing 98% correct and 99.5% complete responses, compared with 89% and 90% for the original creators' answers.

Problem

• programming tutorial videos
• viewer questions
• automated responses
• comment classification

Innovation

• Retrieval-Augmented Generation
• Large Language Models
• Programming Tutorial Videos
• Automated Question Answering
• Comment Classification