An Empirical Study of Multi-Agent RAG for Real-World University Admissions Counseling

2025-07-15

AI Summary

To address high hallucination rates, low accuracy, and high deployment costs of automated higher-education admission counseling systems in real-world settings, this paper proposes MARAUS—a Multi-Agent Retrieval-Augmented system tailored to Vietnamese universities’ practical enrollment needs. Methodologically, MARAUS integrates multi-agent collaborative orchestration with a hybrid retrieval mechanism, implemented atop a RAG framework using GPT-4o mini for lightweight, efficient deployment. Its key contribution lies in establishing a reusable AI counseling paradigm for low-resource educational environments, significantly mitigating hallucinations and enhancing response quality. Empirical evaluation across over 6,000 real-user interactions demonstrates an average accuracy of 92%, a reduction in hallucination rate from 15% to 1.45%, sub-4-second average response latency, and a total deployment cost of only $11.58 over two weeks.

Technology Category

• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval
• Multiagent Systems: Adversarial Agents
• Humans and AI: Voting

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

This paper presents MARAUS (Multi-Agent and Retrieval-Augmented University Admission System), a real-world deployment of a conversational AI platform for higher education admissions counseling in Vietnam. While large language models (LLMs) offer potential for automating advisory tasks, most existing solutions remain limited to prototypes or synthetic benchmarks. MARAUS addresses this gap by combining hybrid retrieval, multi-agent orchestration, and LLM-based generation into a system tailored for real-world university admissions. In collaboration with the University of Transport Technology (UTT) in Hanoi, we conducted a two-phase study involving technical development and real-world evaluation. MARAUS processed over 6,000 actual user interactions, spanning six categories of queries. Results show substantial improvements over LLM-only baselines: on average 92 percent accuracy, hallucination rates reduced from 15 precent to 1.45 percent, and average response times below 4 seconds. The system operated cost-effectively, with a two-week deployment cost of 11.58 USD using GPT-4o mini. This work provides actionable insights for the deployment of agentic RAG systems in low-resource educational settings.

Problem

• Develops AI system for real-world university admissions counseling
• Reduces LLM hallucination rates in educational advisory tasks
• Demonstrates cost-effective multi-agent RAG deployment in low-resource settings

Innovation

• Hybrid retrieval enhances information accuracy
• Multi-agent orchestration improves system efficiency
• LLM-based generation reduces hallucination rates