Evidence-Traceable Dynamic Interviewer Architecture for Expertise-Adaptive Qualitative Interviews Using Local LLMs

2026-10-08

AI Summary

This study addresses the limitations of existing automated interview systems, where fixed question sequences lead to insufficient personalization and redundant inquiries. We propose a dynamic interview architecture based on a local large language model. The architecture incorporates a five-module prompt-driven mechanism with persistent state tracking to assess participant expertise in real time, adaptively adjust question depth, maintain semantic continuity, and ensure evidence-traceable interactions. Experimental results demonstrate that the system achieves 78.9% accuracy in expert profiling, with question complexity significantly correlated with user proficiency levels. Furthermore, it attains a satisfaction score of 4.38 out of 5, effectively validating the feasibility and superiority of this adaptive interviewing paradigm.

Abstract

Automated interviewers and conversational agents are increasingly used in research, recruitment, customer service, and education. However, many existing systems rely on fixed question sequences and provide limited context-based personalization without considering participants' knowledge, which can lead to repetitive or irrelevant follow-up questions. Therefore, there is a need for an adaptive interviewing system that can adjust question depth while maintaining conversational continuity and semantic progression. To address this, an Evidence-Traceable Dynamic Interviewer Architecture is presented using a locally hosted Large Language Model (LLM), with the interview continuously adapted throughout the entire conversation based on the participant's responses and evolving context. The interviewer profiles participants' expertise in real time to generate knowledge-appropriate questions, well-articulated responses, and smooth transition messages that support conversational continuity. A five-module prompt-driven architecture and persistent interview-state record support these functions. The interviewer was evaluated with 246 participants. Expertise Profiling module (M3) showed 78.9% exact agreement with independently reported participant expertise, with a weighted Cohen's K of 0.80. Generate Iterative Questions module (M4) showed a strong expertise-complexity association (p=.79, p<.001), and participants reported high relevance (mean 4.41), engagement (mean 4.32), and satisfaction (mean 4.38), providing evidence that the architecture's adaptive components operated consistently with their intended functions while participants reported a positive interview experience.

Problem

• Adaptive Interviewing
• Expertise Profiling
• Conversational Agents
• Dynamic Question Generation
• Contextual Personalization

Innovation

• Dynamic Interviewer Architecture
• Expertise Profiling
• Local LLMs
• Prompt-driven Architecture
• Adaptive Qualitative Interviews