ReDesign: Recovering Editable Design Structures from Images via Agentic Decomposition

2026-07-28

AI Summary

Recovering editable design files from raster images—requiring joint reconstruction of layout, vector geometry, color, and layer hierarchy—remains a key challenge in design automation. This work proposes an agent-based framework that dynamically orchestrates multimodal specialized tools to incrementally construct hierarchical layers, incorporating a local graceful validation mechanism (accept/prune/retry) at each step to effectively mitigate error accumulation in long tool-use chains. The study introduces the first large-scale benchmark for editability evaluation, the Figma Edit Replay Benchmark, and demonstrates significant improvements over existing methods in both visual fidelity and editability, with particularly strong performance on layout, color, and text editing tasks.

Abstract

Recovering an editable design file from a raster image is a common and costly bottleneck in modern design workflows, yet remains challenging since editability depends on recovering multi-modal attributes, such as typography, vector geometry, colors, grouping, and layer ordering. We present ReDesign, an agentic framework that grows an editable layer hierarchy by selecting and composing specialized tools across modalities. To keep this long decision process reliable despite imperfect tool outputs, we introduce graceful verification at each expansion, which provides local accept, prune, or retry feedback that prevents error accumulation and avoids large scale reruns. To evaluate editability at scale, we introduce the Figma Edit Replay Benchmark, consisting of 909 raw Figma files and 14,796 controlled edit instructions that replay edits on reconstructed outputs. Across this benchmark and standard reconstruction metrics, ReDesign achieves strong visual fidelity while delivering the highest editability across layout, color, and text edits, outperforming layered decomposition baselines and serial tool use pipelines.

Problem

• editable design recovery
• raster image to vector
• multi-modal attribute reconstruction
• design structure recovery
• layer hierarchy

Innovation

• agentic decomposition
• editable design recovery
• graceful verification
• Figma Edit Replay Benchmark
• multi-modal design reconstruction