Learn, Explore and Reflect by Chatting: Understanding the Value of an LLM-Based Voting Advice Application Chatbot

2025-05-14

AI Summary

Existing voting advice applications (VAAs) suffer from linguistic complexity and rigid interaction paradigms, limiting accessibility for low-digital-literacy voters. This study introduces the first large language model (LLM)-integrated VAA chatbot, empirically deployed ahead of the 2024 European Parliament elections in Germany (N=331). A mixed-method evaluation—combining surveys, dialogue log analysis, and in-depth interviews—assesses its impact. Methodologically, the system replaces conventional click-based interfaces with natural-language-driven, personalized learning, proactive exploration, and justification-aware reflection. Its key contributions are threefold: (1) it empirically uncovers LLMs’ catalytic role in enhancing political cognition and deliberative decision-making; (2) it proposes a democratic technology design framework balancing interpretability and trustworthy human-AI interaction; and (3) it demonstrates significant improvements in information efficacy, usability intuitiveness, and reflective depth, alongside high user acceptance and actionable design guidelines for real-world deployment.

Technology Category

• Humans and AI: Voting
• Machine Learning: Large Multimodal Models (LMMs)
• Game Theory and Economic Paradigms: Social Choice / Voting

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Voting advice applications (VAAs), which have become increasingly prominent in European elections, are seen as a successful tool for boosting electorates' political knowledge and engagement. However, VAAs' complex language and rigid presentation constrain their utility to less-sophisticated voters. While previous work enhanced VAAs' click-based interaction with scripted explanations, a conversational chatbot's potential for tailored discussion and deliberate political decision-making remains untapped. Our exploratory mixed-method study investigates how LLM-based chatbots can support voting preparation. We deployed a VAA chatbot to 331 users before Germany's 2024 European Parliament election, gathering insights from surveys, conversation logs, and 10 follow-up interviews. Participants found the VAA chatbot intuitive and informative, citing its simple language and flexible interaction. We further uncovered VAA chatbots' role as a catalyst for reflection and rationalization. Expanding on participants' desire for transparency, we provide design recommendations for building interactive and trustworthy VAA chatbots.

Problem

• Enhancing voter engagement with LLM-based chatbot VAAs
• Addressing complex language in voting advice applications
• Exploring chatbots for tailored political decision-making

Innovation

• LLM-based chatbot for voting advice
• Simple language and flexible interaction
• Design for interactive trustworthy VAAs