A Summary on GUI Agents with Foundation Models Enhanced by Reinforcement Learning

2025-04-29

AI Summary

This paper addresses the weak adaptability, poor generalization, and insufficient long-term stability of multimodal large language model (MLLM)-driven GUI agents in complex real-world environments. We propose a reinforcement learning (RL)-based enhancement framework that formalizes GUI interaction as a Markov decision process and systematically decomposes the agent’s evolution across perception–planning–execution modules. We introduce, for the first time, a three-tiered taxonomy of GUI agent training paradigms—prompt engineering, supervised fine-tuning, and RL—highlighting the critical transition from static response generation to dynamic policy learning. Leveraging GUI environment simulation and modular architecture design, we empirically validate RL’s central role in improving cross-application generalization, robustness, and long-horizon task stability. The work establishes the first comprehensive RL-enhanced theoretical framework and practical guidelines tailored specifically for GUI agents.

Technology Category

• Machine Learning: Reinforcement Learning
• Multiagent Systems: Multiagent Learning
• Cognitive Modeling & Cognitive Systems: Agent Architectures

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information

Abstract

Graphical User Interface (GUI) agents, driven by Multi-modal Large Language Models (MLLMs), have emerged as a promising paradigm for enabling intelligent interaction with digital systems. This paper provides a structured summary of recent advances in GUI agents, focusing on architectures enhanced by Reinforcement Learning (RL). We first formalize GUI agent tasks as Markov Decision Processes and discuss typical execution environments and evaluation metrics. We then review the modular architecture of (M)LLM-based GUI agents, covering Perception, Planning, and Acting modules, and trace their evolution through representative works. Furthermore, we categorize GUI agent training methodologies into Prompt-based, Supervised Fine-Tuning (SFT)-based, and RL-based approaches, highlighting the progression from simple prompt engineering to dynamic policy learning via RL. Our summary illustrates how recent innovations in multimodal perception, decision reasoning, and adaptive action generation have significantly improved the generalization and robustness of GUI agents in complex real-world environments. We conclude by identifying key challenges and future directions for building more capable and reliable GUI agents.

Problem

• Enhancing GUI agents with Reinforcement Learning for better interaction
• Formalizing GUI tasks as Markov Decision Processes for optimization
• Improving generalization and robustness in complex real-world environments

Innovation

• Reinforcement Learning enhances GUI agent architectures
• Multimodal perception improves decision reasoning
• Adaptive action generation boosts agent robustness