OnomaCompass: A Texture Exploration Interface that Shuttles between Words and Images

2026-01-08

arXiv.org

AI Summary

This work addresses the challenge users face in precisely articulating material aesthetics during early-stage design. To overcome this limitation, the authors propose a dual latent-space mapping system that leverages onomatopoeic words and images, introducing sound symbolism—specifically onomatopoeia—as a lightweight semantic cue to replace conventional text prompts for cross-modal affective exploration. The system integrates a Stable Diffusion–generated texture dataset, a custom onomatopoeia–texture pairing corpus, an image-editing model, and video interpolation techniques to enable synchronized word–image browsing, real-time material previewing, and gallery-based curation, thereby forming a closed-loop exploratory workflow. User studies demonstrate that, compared to traditional prompting methods, this approach significantly reduces cognitive load and frustration while enhancing user enjoyment, effectively facilitating the externalization of vague sensory expectations and serendipitous discovery.

Technology Category

• Cognitive Modeling & Cognitive Systems: Affective Computing
• Natural Language Processing: Prompt Engineering / Prompting
• Search and Optimization: Mixed Discrete/Continuous Search

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems
• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data

Abstract

Humans can finely perceive material textures, yet articulating such somatic impressions in words is a cognitive bottleneck in design ideation. We present OnomaCompass, a web-based exploration system that links sound-symbolic onomatopoeia and visual texture representations to support early-stage material discovery. Instead of requiring users to craft precise prompts for generative AI, OnomaCompass provides two coordinated latent-space maps--one for texture images and one for onomatopoeic term--built from an authored dataset of invented onomatopoeia and corresponding textures generated via Stable Diffusion. Users can navigate both spaces, trigger cross-modal highlighting, curate findings in a gallery, and preview textures applied to objects via an image-editing model. The system also supports video interpolation between selected textures and re-embedding of extracted frames to form an emergent exploration loop. We conducted a within-subjects study with 11 participants comparing OnomaCompass to a prompt-based image-generation workflow using Gemini 2.5 Flash Image ("Nano Banana"). OnomaCompass significantly reduced workload (NASA-TLX overall, mental demand, effort, and frustration; p<.05) and increased hedonic user experience (UEQ), while usability (SUS) favored the baseline. Qualitative findings indicate that OnomaCompass helps users externalize vague sensory expectations and promotes serendipitous discovery, but also reveals interaction challenges in spatial navigation. Overall, leveraging sound symbolism as a lightweight cue offers a complementary approach to Kansei-driven material ideation beyond prompt-centric generation.

Problem

• material ideation
• sensory articulation
• design cognition
• texture perception
• sound symbolism

Innovation

• sound symbolism
• latent space navigation
• cross-modal exploration
• generative AI interface
• material ideation