Evaluating Design Video Generation: Metrics for Compositional Fidelity

2026-05-15

AI Summary

This work addresses the lack of standardized, automated evaluation methods for design animation video generation, which hinders objective assessment of generation quality under structured constraints. To bridge this gap, we propose the first multidimensional automatic evaluation framework tailored specifically for design animations. Leveraging computer vision and video analysis techniques, the framework quantifies key generative attributes across four dimensions: layout fidelity, motion correctness, temporal consistency, and content fidelity. Operating without human intervention, it delivers an objective and reproducible benchmark that enables fair comparison among diverse generative models and supports sustained progress in the field.

Abstract

Generative video models are increasingly used in design animation tasks, yet no standardized evaluation framework exists for this domain. Unlike natural video generation, design animation imposes structured constraints: specific components shall animate with prescribed motion types, directions, speed and timing, while non-animated regions must remain stable and layout structure must be preserved. This paper provides a fully automated evaluation framework organized across four dimensions: layout fidelity, motion correctness, temporal quality, and content fidelity. This eliminates the reliance on subjective human evaluation and establishes a common basis for benchmarking progress in the field.

Problem

• design video generation
• evaluation framework
• compositional fidelity
• layout preservation
• motion constraints

Innovation

• design video generation
• compositional fidelity
• automated evaluation
• motion correctness
• layout fidelity