Partnering with Generative AI: Experimental Evaluation of Human-Led and Model-Led Interaction in Human-AI Co-Creation

2025-10-27

AI Summary

This study investigates the design of human–large language model (LLM) collaboration paradigms for creative tasks. Through a randomized controlled experiment, it comparatively evaluates two interaction paradigms—human-led reflective interaction versus model-led proactive rewriting—using quantitative metrics for creativity quality, diversity, and user sense of ownership. The study proposes and empirically validates a novel “reflective human-led interaction” paradigm: users initiate prompts, LLMs respond, and users iteratively reflect and refine outputs—positioning the LLM as a cognitive partner rather than a substitute. Results demonstrate that this paradigm significantly enhances creative quality while preserving diversity and user agency; in contrast, the model-led paradigm improves quality but substantially diminishes both diversity and perceived ownership. The findings provide empirically grounded design principles and theoretical foundations for generative AI interfaces, advancing human-centered co-creation frameworks.

Technology Category

• Cognitive Modeling & Cognitive Systems: Computational Creativity
• Humans and AI: Intelligent User Interfaces
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Economics, Online Markets and Human Computation: Uses of LLMs and GenAI for marketplace design, bidding, and strategic interactions

Abstract

Large language models (LLMs) show strong potential to support creative tasks, but the role of the interface design is poorly understood. In particular, the effect of different modes of collaboration between humans and LLMs on co-creation outcomes is unclear. To test this, we conducted a randomized controlled experiment ($N = 486$) comparing: (a) two variants of reflective, human-led modes in which the LLM elicits elaboration through suggestions or questions, against (b) a proactive, model-led mode in which the LLM independently rewrites ideas. By assessing the effects on idea quality, diversity, and perceived ownership, we found that the model-led mode substantially improved idea quality but reduced idea diversity and users' perceived idea ownership. The reflective, human-led mode also improved idea quality, yet while preserving diversity and ownership. Our findings highlight the importance of designing interactions with generative AI systems as reflective thought partners that complement human strengths and augment creative processes.

Problem

• Evaluating human-led versus model-led interaction modes in human-AI co-creation
• Assessing how collaboration modes affect idea quality, diversity, and ownership
• Determining optimal AI interaction designs to augment human creativity

Innovation

• Evaluated human-led versus model-led interaction modes
• Model-led mode improved quality but reduced diversity
• Human-led mode enhanced quality while preserving ownership