TestAgent: An Adaptive and Intelligent Expert for Human Assessment

2025-06-03

AI Summary

Existing adaptive assessment methods face three critical bottlenecks: (1) mechanistic item selection that encourages guessing, (2) inability to handle open-ended responses, and (3) subjective evaluation constrained by noisy responses and coarse-grained outputs. This paper proposes the first large language model (LLM)-driven adaptive assessment agent, which enables personalized item generation, context-aware real-time response analysis, and fine-grained latent state modeling via dynamic multi-turn dialogue—augmented with anti-guessing mechanisms and response anomaly detection. Key innovations include: (1) the first deep integration of LLMs into an adaptive assessment framework; (2) semantic understanding and progressive evaluation of open-ended responses; and (3) elimination of static item banks and fixed selection policies. Experiments across psychological, educational, and lifestyle assessment tasks demonstrate superior accuracy over SOTA methods, a 20% reduction in required items, and significant improvements in assessment speed, interaction fluency, and user satisfaction.

Technology Category

Machine Learning: Large Multimodal Models (LMMs)Cognitive Modeling & Cognitive Systems: Adaptive BehaviorMultiagent Systems: Adversarial Agents

Application Category

User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendationSemantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactionsSearch and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information

Abstract

Accurately assessing internal human states is key to understanding preferences, offering personalized services, and identifying challenges in real-world applications. Originating from psychometrics, adaptive testing has become the mainstream method for human measurement and has now been widely applied in education, healthcare, sports, and sociology. It customizes assessments by selecting the fewest test questions . However, current adaptive testing methods face several challenges. The mechanized nature of most algorithms leads to guessing behavior and difficulties with open-ended questions. Additionally, subjective assessments suffer from noisy response data and coarse-grained test outputs, further limiting their effectiveness. To move closer to an ideal adaptive testing process, we propose TestAgent, a large language model (LLM)-powered agent designed to enhance adaptive testing through interactive engagement. This is the first application of LLMs in adaptive testing. TestAgent supports personalized question selection, captures test-takers' responses and anomalies, and provides precise outcomes through dynamic, conversational interactions. Experiments on psychological, educational, and lifestyle assessments show our approach achieves more accurate results with 20% fewer questions than state-of-the-art baselines, and testers preferred it in speed, smoothness, and other dimensions.

Problem

• Enhancing adaptive testing accuracy with interactive LLM-powered agents
• Reducing guessing behavior and handling open-ended questions effectively
• Improving personalized assessments with fewer questions and dynamic interactions

Innovation

• LLM-powered agent for adaptive testing
• Personalized question selection via interaction
• Dynamic conversational engagement reduces questions