CUA-Sandbox: Efficient Environments for Computer-Use Agent Reinforcement Learning

2026-09-26

AI Summary

This study addresses the substantial memory and initialization overhead caused by conventional independent runtime duplication in reinforcement learning for computer-use agents. To overcome this, it proposes treating state, rather than runtime, as the fundamental unit of environment independence. Methodologically, the work introduces a decoupled architecture comprising state capsules and shared runtimes alongside a reset-branching mechanism. By integrating state-scoped execution, transactional lifecycle management, and containerized isolation, the approach enables efficient parallel environment simulation while preserving native software interfaces. Experimental results demonstrate that, compared to Docker-based baselines, the proposed method achieves a 6.2× increase in throughput, a 9.2× reduction in per-environment memory consumption, and a 504× decrease in incremental storage, all while maintaining comparable or superior task success rates.

Abstract

Reinforcement learning enables computer-use agents to improve through interaction with real software environments, including websites and desktop applications. However, conventional deployments replicate an initialized runtime for each independent rollout, even when trajectories use the same software, incurring repeated memory and initialization costs as the number of parallel environments grows. Does an independent computer-use environment require an independent execution runtime? Our key observation is that trajectories require independent mutable state, while initialized application runtimes can be reused across concurrently evolving environments, making state the natural unit of environment independence. Guided by this observation, we introduce CUA-Sandbox, which separates private state capsules from shared runtimes through state-scoped execution and transactional lifecycle operations, including resets and branches, while retaining the original software interfaces and task evaluators. Experiments show comparable or improved task success relative to Docker, while substantially reducing rollout and resource costs. CUA-Sandbox achieves up to a 6.20x increase in rollout throughput, a 9.2x reduction in per-environment memory, and a 504x reduction in incremental storage.

Problem

• Computer-Use Agent
• Reinforcement Learning
• Environment Efficiency
• Resource Overhead
• Parallel Rollouts

Innovation

• Computer-Use Agent
• Reinforcement Learning
• State-Scoped Execution
• Shared Runtime
• Resource Efficiency