sketch-plot: Progressive Editing for Text-to-Image Academic Figures

2026-06-08

AI Summary

Current text-to-image models produce academic figures as non-editable bitmaps, necessitating full regeneration for any local modification—a process that often disrupts already satisfactory content. To address this limitation, this work proposes a three-tier progressive editing system that begins with a generated PNG image and automatically decomposes it into addressable tiles, selectively converting them into segmented SVG representations to enable user edits at multiple granularities. This approach introduces the first on-demand, tile-based controllable editing mechanism tailored for AI-generated academic figures, enhanced by a human-in-the-loop interface that improves the accuracy of semantic segmentation and vectorization. User studies demonstrate that the proposed method significantly outperforms full-regeneration workflows, with domain experts highly endorsing its efficiency and precision in targeted figure editing.

Abstract

Text to image (T2I) models such as gpt-image-2 can now generate publication grade academic figures from a short prompt, but the output is a flat raster: a user who wants to change one arrow, one label, or one icon has to regenerate the whole image, which also disturbs the parts they wanted to keep. We present sketch-plot, an interactive system that closes this controllability gap with a three layer progressive editing pipeline: a generated PNG, an addressable puzzle of editable pieces, and a per piece SVG. The user stops at the layer that gives them enough control for the change at hand, so the cost of decomposition and vectorisation is paid only on the pieces that need it. Realising this pipeline is not trivial. General segmentation models lack the semantic discriminability to decompose a research figure cleanly, and end to end image vectorisation produces incomplete shapes and loses semantic structure. We therefore route both stages through a human in the loop interface that lets the user accept, refine, or reject decomposition and vectorisation decisions on a piece by piece basis. We validate the design with an expert user study, in which participants found sketch-plot effective for making targeted edits to AI generated academic figures and preferred it over regenerating the whole image. A demonstration video is available at https://anonymous.4open.science/r/SketchPlotVideo/.

Problem

• text-to-image
• academic figures
• controllable editing
• image decomposition
• vectorisation

Innovation

• progressive editing
• text-to-image
• vectorization
• human-in-the-loop
• academic figures