Recipes for Creativity: Iterative Generation and Evaluation in Large Language Models

2026-08-07

AI Summary

Traditional generative models struggle to emulate the human process of iteratively refining creative ideas. This work adapts the FunSearch framework by integrating an iterative generate-and-select mechanism within large language models (LLMs), using recipe generation as a testbed, and constructs an automated LLM-based evaluation system grounded in the Torrance Tests of Creative Thinking (TTCT). The study reveals that the design of the in-the-loop evaluator is more critical than the number of iterations; notably, lightweight scoring models significantly outperform larger counterparts across most TTCT dimensions. Moreover, the creativity of the generated recipes rivals that of human benchmarks. These findings underscore the pivotal role of the evaluation mechanism in enabling efficient and effective creative generation.

Abstract

Generative models are often evaluated through singular artifacts, whereas human creativity typically emerges through iterative generation, appraisal, and refinement. This pilot study examines whether iterative search improves LLM creativity by adapting FunSearch to recipe generation for the 2024 Pillsbury Bake-Off and evaluating outputs against human benchmarks using TTCT-based LLM evaluation. Across two experiments, we test iteration count, generator temperature, and in-loop selection-scorer model size. Results show that iterative generation-selection can produce recipes with creativity scores comparable to human benchmarks, but additional iterations alone do not improve creativity. The in-loop evaluator matters most: a smaller selection scorer yields significantly higher scores across most TTCT dimensions, while temperature has limited effects except for originality. These findings suggest that evaluator design is a first-order design variable in subjective creative search.

Problem

• creativity
• iterative generation
• large language models
• recipe generation
• evaluation

Innovation

• iterative generation
• in-loop evaluation
• LLM creativity
• FunSearch
• TTCT-based evaluation