Generative AI Voting: Fair Collective Choice is Resilient to LLM Biases and Inconsistencies

2024-05-31

arXiv.org

AI Summary

This work investigates fairness and representativeness when large language models (LLMs) serve as proxies for human voters in collective decision-making. We construct over 50,000 LLM-based voter personas and simulate their behavior across 306 real-world elections. For the first time, we apply proportional representation aggregation methods—such as Equal Shares—to LLM-based deliberative settings. By comparing complex preference voting against plurality rule and integrating multiple model generations (e.g., GPT-3/3.5, Llama2), we identify significant preference inconsistency among LLMs; yet fair aggregation yields highly representative outcomes. Notably, it robustly recovers representation for abstainers, enhancing democratic resilience under low turnout. Our core contribution is the empirical validation of AI-mediated representation feasibility and the demonstration that fairness-aware aggregation algorithms simultaneously improve fairness for both human constituents and AI proxies—a synergistic fairness gain.

Technology Category

• Humans and AI: Voting
• Knowledge Representation and Reasoning: Preferences
• Game Theory and Economic Paradigms: Social Choice / Voting

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Scaling up deliberative and voting participation is a longstanding endeavor -- a cornerstone for direct democracy and legitimate collective choice. Recent breakthroughs in generative artificial intelligence (AI) and large language models (LLMs) unravel new capabilities for AI personal assistants to overcome cognitive bandwidth limitations of humans, providing decision support or even direct representation of human voters at large scale. However, the quality of this representation and what underlying biases manifest when delegating collective decision-making to LLMs is an alarming and timely challenge to tackle. By rigorously emulating with high realism more than>50K LLM voting personas in 306 real-world voting elections, we disentangle the nature of different biases in LLMS (GPT 3, GPT 3.5, and Llama2). Complex preferential ballot formats exhibit significant inconsistencies compared to simpler majoritarian elections that show higher consistency. Strikingly though, by demonstrating for the first time in real-world a proportional representation of voters in direct democracy, we are also able to show that fair ballot aggregation methods, such as equal shares, prove to be a win-win: fairer voting outcomes for humans with fairer AI representation, especially for voters who are likely to abstain. This novel underlying relationship proves paramount for democratic resilience in progressives scenarios with low voters turnout and voter fatigue supported by AI representatives: abstained voters are mitigated by recovering highly representative voting outcomes that are fairer. These interdisciplinary insights provide remarkable foundations for science, policymakers, and citizens to develop safeguards and resilience for AI risks in democratic innovations.

Problem

• Investigates biases in LLMs for collective decision-making representation
• Analyzes inconsistencies in LLM voting across different election formats
• Demonstrates fair ballot aggregation improves AI-human voting outcomes

Innovation

• Emulating 50K LLM voting personas realistically
• Analyzing biases in GPT and Llama2 models
• Using fair ballot aggregation for proportional representation