Human Creativity in the Age of LLMs: Randomized Experiments on Divergent and Convergent Thinking

2024-09-24

arXiv.org

AI Summary

This study investigates whether large language model (LLM)-assisted creative writing undermines human independent creativity. Method: Two pre-registered randomized controlled trials (N = 1,100) employed standardized assessments—Alternative Uses Test (AUT) and Remote Associates Test (RAT)—to compare the short-term gains and long-term effects of two LLM intervention types—answer-provision versus Socratic prompting—on divergent and convergent thinking. Contribution/Results: While LLM assistance significantly improved immediate task performance, all assisted groups exhibited significantly lower creative output quantity and quality than the control group in a final AI-free assessment phase—providing the first empirical evidence of “creativity dependency risk”: non-coaching LLM support may impair sustained autonomous creative capacity. These findings challenge the default assumption that AI augmentation invariably enhances cognition, advance a novel theoretical proposition on cognitive dependency, and offer critical cognitive-science foundations for designing human-AI collaborative systems.

Technology Category

Cognitive Modeling & Cognitive Systems: Computational CreativityMachine Learning: Large Multimodal Models (LMMs)Humans and AI: Game Design — Virtual Humans, NPCs and Autonomous Characters

Application Category

Economics, Online Markets and Human Computation: LLM based quality controls for crowd workSemantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMsUser Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Large language models are transforming the creative process by offering unprecedented capabilities to algorithmically generate ideas. While these tools can enhance human creativity when people co-create with them, it's unclear how this will impact unassisted human creativity. We conducted two large pre-registered parallel experiments involving 1,100 participants attempting tasks targeting the two core components of creativity, divergent and convergent thinking. We compare the effects of two forms of large language model (LLM) assistance -- a standard LLM providing direct answers and a coach-like LLM offering guidance -- with a control group receiving no AI assistance, and focus particularly on how all groups perform in a final, unassisted stage. Our findings reveal that while LLM assistance can provide short-term boosts in creativity during assisted tasks, it may inadvertently hinder independent creative performance when users work without assistance, raising concerns about the long-term impact on human creativity and cognition.

Problem

• Impact of LLMs on human creativity
• Effects of LLM assistance on unassisted tasks
• Long-term influence on independent creative performance

Innovation

• LLMs enhance human creativity
• Two forms of LLM assistance tested
• LLMs may hinder independent creativity