Bunraku: Turning a Single Illustration into an Editable Live2D Character

2026-07-29

AI Summary

This work addresses the labor-intensive process of traditional Live2D character creation, which relies on manual layer segmentation, occlusion inpainting, and keyframe specification, lacking an end-to-end method for generating structured assets. We propose the first system capable of automatically producing a complete Live2D character from a single illustration, yielding ordered RGBA layers, deformation meshes, and parameter-driven vertex offsets for key poses. Our approach operates in two stages: layered diffusion-based decomposition followed by cross-layer joint mesh animation generation. Key innovations include the first end-to-end pipeline for structured Live2D asset synthesis, a displacement prediction mechanism leveraging cross-layer self-attention and direction–log-magnitude decomposition to ensure animation consistency, and support for text-guided texture editing while preserving original animations. Experiments show that Stage 2 achieves a mean vertex direction cosine of 0.768 (median 0.828) without teacher forcing, and we release the Live2D-Bench benchmark along with a supervised corpus of 8,884 models.

Abstract

Live2D is the dominant 2D character-animation format for anime characters and virtual avatars, representing each character as a stack of RGBA layers driven by per-layer mesh deformation. Despite its wide use in virtual streaming, mobile games, and interactive characters, authoring a Live2D model still demands weeks of manual layer separation, occlusion completion, mesh placement, and keyframing, and no prior generative method produces such a structured asset end-to-end. We present the first system that, from a single illustration, generates all the structured information a Live2D runtime consumes: ordered RGBA layers, a deformation mesh per layer, and the parameter-driven keypose vertex offsets that make the character move. Stage 1 casts layered decomposition as a layered diffusion process under a Live2D-aware organ-level taxonomy, producing an ordered RGBA stack with hidden-region completion. Stage 2 builds a content-conforming triangle mesh for each layer from its alpha channel alone, then predicts the keypose displacement field of all layers jointly: every vertex of every layer is one token, self-attention spans layer boundaries, and each displacement is factorised into a bounded direction and a log-magnitude. Joint rather than independent prediction is what makes the result a coherent character instead of separately plausible parts, and is our largest gain; scaling the network 112x yields none. On 50 held-out characters, under true generation with no teacher forcing, Stage 2 attains a per-vertex direction cosine of 0.768 (median 0.828). Because a layer's mesh derives from its alpha channel, a clothing layer can be re-textured from a natural-language instruction while the mesh and predicted animation are reused byte-for-byte. We further contribute Live2D-Bench, the first standardized benchmark for the task, and an 8,884-model Live2D corpus with layer and animation supervision.

Problem

• Live2D
• layered decomposition
• character animation
• single-image generation
• structured asset generation

Innovation

• layered diffusion
• cross-layer joint prediction
• Live2D generation
• content-conforming mesh
• structured animation synthesis