JarvisHub: An Open Harness for Canvas-Native Multimodal Creative Agents

2026-07-26

AI Summary

This work addresses the challenge of effectively managing dynamic project states—such as assets, drafts, versions, and feedback—in long-horizon, multimodal creative workflows with current generative AI systems. The authors propose an open framework centered on an editable canvas, which uniquely unifies the roles of workspace, external memory, and action space. By representing assets, dependencies, and feedback through a node-based multimodal structure, and integrating a three-layer architecture—comprising a canvas state layer, a protocol bridging layer, and an agent runtime—the framework enables synchronized state management and tool invocation. This design allows AI agents to continuously plan and generate within a shared, inspectable, and intervenable state, empowering users to actively participate throughout the creative process and shifting creative AI from single-step generation toward a traceable, editable, and collaborative paradigm for sustained creation.

Abstract

Creative AI is moving from single-step asset generation toward long-horizon multimodal production. Although recent generative models can synthesize high-quality images, videos, audio clips, UI elements, storyboards, slides, and other creative assets, real-world creative work requires more than isolated prompt-output interactions. It involves references, drafts, alternatives, edits, failed attempts, version relations, tool actions, evaluation signals, and human feedback, which together form an evolving project state. Existing prompt-based, chat-based, and node-based generation systems only partially support this state, as they often discard intermediate context, rely on linear conversations, or require manually specified workflows. Recent commercial systems indicate a shift toward agent-assisted creative production, but their closed architectures make it difficult to study how agents represent context, choose tools, revise artifacts, recover from failures, and maintain consistency over time. To address this gap, we introduce JarvisHub, a canvas-native creative agent harness for long-horizon multimodal creation. JarvisHub treats an editable canvas as the user workspace, the agent's external memory, action space, and shared project state, representing multimodal artifacts, dependencies, versions, and feedback as typed canvas nodes and links. Through a three-layer architecture of canvas state, protocol bridge, and agent runtime, JarvisHub enables agents to act within an inspectable and editable creative state. This design moves creative agents beyond isolated tool use toward sustained, human-steerable creative automation, where agents can progressively plan, generate, revise, and organize multimodal projects while users remain able to inspect, guide, and intervene throughout the process.

Problem

• multimodal creative agents
• long-horizon creation
• project state
• canvas-native interface
• agent-assisted production

Innovation

• canvas-native
• multimodal creative agents
• long-horizon creation
• external memory
• human-steerable automation