MAGA: Multi-Platform Self-Fusion of GUI Agents via Structured Action Distillation

2026-07-31

AI Summary

Current large language model–driven GUI agents are often confined to a single platform, hindering unified deployment across mobile, web, and desktop environments. This work proposes a structured action distillation framework that consolidates multiple domain-specific teacher models into a single cross-platform student policy. The approach employs an action-aware signal reweighting mechanism to dynamically suppress ineffective distillation signals and prioritize learning from erroneous actions. Additionally, it introduces a training-only prompt optimization technique that enhances the quality of teacher supervision without altering the student’s input format. Experimental results demonstrate that the method achieves state-of-the-art average success rates across two model scales, with the 8B variant outperforming the strongest baseline by 2.0% and approaching the average performance of the domain-specific teacher models.

Abstract

Graphical user interface (GUI) agents based on large language models are increasingly deployed across mobile, web, and desktop environments. However, existing agents are typically domain-specific, limiting the deployment and user experience. This motivates the consolidation of specialized models into a single cross-environment policy. Weight merging directly merges domain-specific experts but can corrupt executable actions under expert disagreement, while on-policy distillation (OPD) avoids conflicting teacher supervision yet still treats all response tokens equally during distillation, ignoring that action tokens are the only interface between the environment and the agent. To address this, We introduce MAGA that re-allocates training signal according to the structured action. Based on the correctness of the generated action, it suppresses unnecessary or invalid distillation signals and focuses learning on erroneous actions. Besides, a training-only hint optimizes the supervision signal provided by domain-specific teachers without changing the student input. Across two model scales, MAGA achieves the highest mean success rate, outperforming the strongest baseline by 2.0% at 8B and achieves almost the same average performance with teachers.

Problem

• GUI agents
• cross-platform
• action distillation
• domain-specific models
• structured action

Innovation

• structured action distillation
• cross-platform GUI agents
• action-aware distillation
• multi-platform self-fusion
• training-only hint