Grammar-based Game Description Generation using Large Language Models

2024-07-24

IEEE Transactions on Games

AI Summary

To address the high syntactic error rate and unreliable end-to-end generation in natural language-to-Game Description Language (GDL) translation, this paper proposes a two-stage LLM-driven framework: first generating a minimal syntactically compliant GDL skeleton, then iteratively refining it under guidance from a customized GDL parser. The method innovatively integrates progressive syntax generation with subsequence-level validity verification, enabling controllable, grammar-constrained generation. Experiments across multiple logic games demonstrate substantial improvements over direct LLM prompting baselines, with a significant increase in GDL syntactic correctness. The implementation is publicly available.

Technology Category

• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Humans and AI: Game Design — Procedural Content Generation & Storytelling
• Knowledge Representation and Reasoning: Description Logics

Application Category

• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Game Description Language (GDL) provides a standardized way to express diverse games in a machine-readable format, enabling automated game simulation, and evaluation. While previous research has explored game description generation using search-based methods, generating GDL descriptions from natural language remains a challenging task. This paper presents a novel framework that leverages Large Language Models (LLMs) to generate grammatically accurate game descriptions from natural language. Our approach consists of two stages: first, we gradually generate a minimal grammar based on GDL specifications; second, we iteratively improve the game description through grammar-guided generation. Our framework employs a specialized parser that identifies valid subsequences and candidate symbols from LLM responses, enabling gradual refinement of the output to ensure grammatical correctness. Experimental results demonstrate that our iterative improvement approach significantly outperforms baseline methods that directly use LLM outputs. Our code is available at https://github.com/tsunehiko/ggdg

Problem

• Large Language Models
• Game Description Language
• Natural Language Generation

Innovation

• Large Language Models
• Iterative Refinement
• Game Description Language