AGOCS -- Accurate Google Cloud Simulator Framework

2025-09-30

AI Summary

To address the lack of low-cost, high-fidelity simulation tools for cloud computing research, this paper proposes a lightweight, modular cloud workload simulator. The simulator leverages real trace data from Google’s 12.5K-node cluster and is optimized to run efficiently on commodity desktop machines. Implemented in Scala, it employs parallelized trace parsing and event-driven simulation to accurately model job-, task-, and node-level behaviors, including dynamic resource scheduling and fine-grained resource utilization patterns. Its core contribution lies in a novel simulation framework that jointly achieves high fidelity—preserving key statistical properties of real traces—and scalability—significantly reducing computational overhead. The framework is open-sourced, providing researchers with a reliable, accessible experimental platform for cloud scheduling, performance analysis, and system optimization.

Technology Category

Search and Optimization: Sampling/Simulation-based SearchCognitive Modeling & Cognitive Systems: Simulating Human BehaviorMachine Learning: Hardware-aware ML

Application Category

Web Mining and Content Analysis: Web data generation and simulationEconomics, Online Markets and Human Computation: Economic aspects and information design of Web, GenAI, and cloud computingUser Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems

Abstract

This paper presents the Accurate Google Cloud Simulator (AGOCS) - a novel high-fidelity Cloud workload simulator based on parsing real workload traces, which can be conveniently used on a desktop machine for day-to-day research. Our simulation is based on real-world workload traces from a Google Cluster with 12.5K nodes, over a period of a calendar month. The framework is able to reveal very precise and detailed parameters of the executed jobs, tasks and nodes as well as to provide actual resource usage statistics. The system has been implemented in Scala language with focus on parallel execution and an easy-to-extend design concept. The paper presents the detailed structural framework for AGOCS and discusses our main design decisions, whilst also suggesting alternative and possibly performance enhancing future approaches. The framework is available via the Open Source GitHub repository.

Problem

• Simulates Google Cloud workloads using real cluster traces
• Reveals precise job and task parameters with resource statistics
• Provides desktop-based high-fidelity cloud simulation for research

Innovation

• Simulator uses real Google Cluster workload traces
• Framework provides precise job and resource statistics
• System implements parallel execution in Scala