STRATA: Self-Learning Through Role-Aligned Tiered Agents for Real-Time Strategy Games

2026-09-29

AI Summary

This study addresses the limitations of large language models in real-time strategy games, including high decision latency, reliance on manual prompting, and lack of continuous learning. To overcome these challenges, this work proposes a role-aligned hierarchical multi-agent architecture that decouples strategic, logistical, and tactical decision-making. Furthermore, it introduces a review agent that innovatively leverages evidence from subsequent matches to verify and compress historical trajectories, generating retrievable experience cards to enable cross-match self-evolution. Experimental results demonstrate that the proposed system increases the win rate from 30% to 100% in fixed scenarios and adaptively acquires differentiated long-term strategic experience tailored to distinct opponent styles.

Abstract

Real-time strategy (RTS) games require agents to coordinate economic development, production and construction, base defense, unit organization, and attack timing over long matches. Existing studies have applied large language models to command decision-making in RTS games, enabling agents to read textual game states and generate high-level plans. However, long inference latency can cause them to miss critical tactical events. The complexity and tactical diversity of full RTS matches also leave existing systems heavily dependent on manually written experience-based prompts, with limited ability to learn continuously from past games. We present STRATA, a role-aligned hierarchical system with cross-game self-learning for Red Alert. STRATA assigns in-game strategic, logistical, and tactical decisions to a Strategic Agent (SA), Logistics Agent (LA), and Tactical Agent (TA), respectively. The SA generates high-level directives based on the global game state and relevant experience cards, while the LA and TA handle logistics and tactical execution. After each match, a Review Agent (RA) derives candidate experience from game traces, validates and revises it using evidence from subsequent matches, and compresses strategic experience supported across multiple games into concise experience cards for SA retrieval. We evaluate STRATA through the formation of experience cards, full-match comparisons before and after learning, and experience learning against AI opponents with different play styles. Under a fixed scenario, using the learned experience cards increases the observed win rate from 30% to 100%. Sequential learning against AI opponents with different play styles also produces distinct long-term strategic experience.

Problem

• Real-Time Strategy Games
• Large Language Models
• Inference Latency
• Continuous Learning
• Decision-Making

Innovation

• Hierarchical Agents
• Self-Learning
• Real-Time Strategy Games
• Large Language Models
• Experience Cards