AI Telephone Surveying: Automating Quantitative Data Collection with an AI Interviewer

2025-07-23

AI Summary

This study addresses the longstanding trade-off in telephone surveying between operational efficiency, interactive flexibility, and methodological rigor. We propose the first generative AI–based standardized quantitative telephone survey system. Methodologically, it integrates large language models (LLMs), automatic speech recognition (ASR), and text-to-speech (TTS) to enable real-time, naturalistic dialogue management—overcoming the limitations of traditional IVR systems by robustly handling interruptions, corrections, and other human conversational behaviors, while strictly enforcing questionnaire logic, item/answer randomization, and linguistic standardization. Our key contribution is the first systematic application of generative AI to methodologically rigorous quantitative survey research. Two pilot studies conducted with the SSRS panel demonstrate that the AI interviewer significantly improves completion rates, reduces breakoff rates, and enhances respondent satisfaction—validating a viable pathway toward reconciling automation with human-centered design, and efficiency with scientific validity.

Technology Category

• Humans and AI: Interaction Techniques and Devices
• Natural Language Processing: Generation
• Multiagent Systems: Agent Communication

Application Category

• Search and Retrieval-Augmented AI: Assisted, interactive, and conversational search
• Social Networks and Social Media: Generative AI / large language models and their impact on social systems
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

With the rise of voice-enabled artificial intelligence (AI) systems, quantitative survey researchers have access to a new data-collection mode: AI telephone surveying. By using AI to conduct phone interviews, researchers can scale quantitative studies while balancing the dual goals of human-like interactivity and methodological rigor. Unlike earlier efforts that used interactive voice response (IVR) technology to automate these surveys, voice AI enables a more natural and adaptive respondent experience as it is more robust to interruptions, corrections, and other idiosyncrasies of human speech. We built and tested an AI system to conduct quantitative surveys based on large language models (LLM), automatic speech recognition (ASR), and speech synthesis technologies. The system was specifically designed for quantitative research, and strictly adhered to research best practices like question order randomization, answer order randomization, and exact wording. To validate the system's effectiveness, we deployed it to conduct two pilot surveys with the SSRS Opinion Panel and followed-up with a separate human-administered survey to assess respondent experiences. We measured three key metrics: the survey completion rates, break-off rates, and respondent satisfaction scores. Our results suggest that shorter instruments and more responsive AI interviewers may contribute to improvements across all three metrics studied.

Problem

• Automating phone surveys using AI for scalable data collection
• Enhancing respondent experience with natural voice AI interactions
• Validating AI survey effectiveness through completion and satisfaction metrics

Innovation

• AI telephone surveying using LLM and ASR
• Natural adaptive respondent experience with voice AI
• Strict adherence to quantitative research best practices