Evolving to the Aesthetics of a Vision-Language Model

2026-05-27

AI Summary

This study addresses the challenge of aesthetic evaluation in evolutionary generative systems for abstract visual art by proposing a novel fitness function that integrates vision-language models (VLMs) with user-guided prompting. The approach first employs CLIP-IQA for initial quality assessment and then introduces a VLM-based pairwise comparison mechanism driven by user-defined prompts. Aesthetic rankings of the generated population are derived from these comparisons using the Glicko rating system. This framework represents the first application of customizable, prompt-driven VLMs to aesthetic evaluation in evolutionary design. The resulting rankings demonstrate strong alignment with expert artists’ subjective judgments, significantly outperforming conventional evaluation methods, and have received positive feedback in practical creative applications.

Abstract

Evolutionary systems have demonstrated remarkable results in creative domains, with recent applications in generative typography, design, and music. However, an open problem remains in designing fitness functions that effectively capture the desired aesthetics of abstract outputs. In this work, we explore two methods for evaluating the aesthetics of a population using Vision-Language Models (VLMs). The first method uses CLIP-IQA to predict an aesthetic score for each design. The second method instead pits candidates against each other, with winners determined by a VLM using a custom prompt specified by the user. The outcomes of these pairwise comparisons are then used to estimate a population ranking via the Glicko rating system. We present these methods in the context of a case study using a custom generative system and compare the resulting rankings with an artist's aesthetic ranking and those produced by other aesthetic evaluation techniques. Additionally, we document the artist's experience using these approaches to evolve designs, critically analysing the strengths and weaknesses of both methods.

Problem

• fitness function
• aesthetics
• abstract outputs
• evolutionary systems
• Vision-Language Models

Innovation

• Vision-Language Models
• Aesthetic Evaluation
• Evolutionary Design
• CLIP-IQA
• Glicko Rating System