Problem
Research questions and friction points this paper is trying to address.
Multimodal AI
Meaning-Making
Reflection
User Authority
Generative AI
"I'm Very Happy for It to Start Hallucinating a Little Bit": Using ClayFlect to Negotiate Multimodal AI Representations in Material Meaning-Making
🤖 AI Summary
This study addresses the challenge of preserving users’ interpretive authority when artificial intelligence participates in personal meaning-making. To this end, it proposes ClayFlect, a system integrating tactile clay interaction with multimodal large language model (MLLM) generation. By framing AI representations as negotiable, provisional states, the system enables users to dynamically construct meaning across material manipulation, dialogue, and generated content. The work validates a “parallel locus of control” mechanism that sustains reflective flow, allowing AI assistance to engage or recede responsively as user reflection deepens. This design facilitates meaning evolution independent of AI mediation, offering a novel paradigm for human-centered, AI-collaborative reflection.
📝 Abstract
As AI enters reflection and emotional support, understanding how it can participate in personal meaning-making while preserving users' authority over interpretation is increasingly important. We present ClayFlect, a novel MLLM-powered system integrating tactile clay-making with conversational and visual generative AI, and report a mixed-methods study with 50 participants. Reflection developed across material, conversational, and generated forms rather than through AI interaction alone. Participants treated AI representations as provisional: they compared, redirected, reinterpreted, selectively incorporated, or left them aside as their artefacts and meanings evolved. Clay provided a directly manipulable space in which participants could continue developing meaning independently of the AI, while generated representations externalised possibilities beyond what they could readily make or visualise. We show how generative AI can participate through representations that remain negotiable, and derive implications for supporting movement across representations, preserving parallel sites of control, and allowing AI support to recede or deepen as reflection unfolds.
Innovation
Methods, ideas, or system contributions that make the work stand out.
Multimodal Large Language Model
Material Meaning-Making
Generative AI
Human-AI Interaction
Tangible Computing
🔎 Similar Papers
No similar papers found.
