Problem
Research questions and friction points this paper is trying to address.
Generative AI
User Feedback
Post-deployment
Feedback Mechanism Design
Human-AI Interaction
Designing the Future of User Feedback for Generative AI
🤖 AI Summary
This study addresses the lack of design guidance in existing generative AI user feedback mechanisms, which results in poor usability and difficulty translating collected data into actionable product optimization insights. Employing a multi-stage empirical methodology, this work first conducts a benchmark evaluation to characterize current industry practices. It then proposes best practices for feedback collection that reconcile user value with data utility, and accordingly designs and evaluates a novel prototype tool. This research bridges a critical theoretical gap in GenAI feedback design by delivering an efficient and flexible feedback experience for users while enabling product teams to acquire structured, actionable performance metrics. Ultimately, it effectively closes the divide between user experience and data utility in generative AI systems.
📝 Abstract
Post-deployment feedback from users can be a cost-effective, scalable, and representative means to monitor and improve generative AI systems and features. When implemented effectively, giving such feedback can increase users' engagement with and trust in GenAI systems. Government regulations and industry guidelines call for post-deployment user engagement, but there is little guidance on designing mechanisms that are usable for consumers and provide actionable input for product teams. We conducted a multi-phase study as a collaboration between academic researchers and eBay. Our benchmark evaluation of current industry approaches identified common issues including lack of discoverability, unclear terminology, and inattention to user value. Based on these findings, we developed best-practice recommendations and designed and tested a prototype feedback-collection tool. The tool aimed to provide users with an efficient, flexible, and positive feedback-giving experience, and provide product teams with rich data on performance and potential problems in a usable format.
Innovation
Methods, ideas, or system contributions that make the work stand out.
Generative AI
User Feedback
Post-deployment Evaluation
Human-Computer Interaction
Feedback Mechanism Design
🔎 Similar Papers
No similar papers found.
💼 Related Jobs
No related jobs found.
