Thinkink: 2D Spatial Ink-native Interaction with LLMs

2026-07-23

AI Summary

This work explores how to naturally integrate large language models (LLMs) into users’ creative ideation processes involving handwritten notes and sketches. To this end, we propose an “ink-native” LLM interaction paradigm and implement a system that accepts handwritten and sketched inputs and spatially integrates LLM responses as ink-like elements on a shared two-dimensional canvas. The system parses ink inputs via semantic tree representations, employs a lightweight state-machine-driven interface for explicit user control, and renders generated content with spatial layout awareness. A three-phase user study (N=12, 6, 10) demonstrates the system’s usability in authentic creative scenarios and yields core design principles for ink-native LLM interactions.

Abstract

People often use handwritten notes and sketches to externalize ideas for ideation. To integrate large language models (LLMs) into this practice, we propose Thinkink. Prompts can be handwritten text or drawn sketches with LLM-generated responses visualized as ink-like text and sketches spatially integrated into a shared canvas. A semantic tree streamlines ink interpretation, and a lightweight UI provides explicit control using a state machine. The tool was designed using a three-stage process. A formative study (N=12) examined current practices with conventional and digital inking methods. The results informed a technical probe for a diagnostic study (N=6) identifying usability and human-LLM interaction challenges. This motivated the design of Thinkink, with a final study (N=10) examining how people incorporate it into their ideation practices. We contribute design implications and a tool for ink-native LLM interaction where the user and LLM write and draw in a shared 2D canvas.

Problem

• ink-native interaction
• large language models
• 2D spatial interaction
• ideation
• human-AI collaboration

Innovation

• ink-native interaction
• spatial LLM integration
• semantic tree
• shared 2D canvas
• handwriting and sketch input