Optimizing CPU Cache Utilization in Cloud VMs with Accurate Cache Abstraction

2025-11-13

AI Summary

Public cloud virtual machines (VMs) lack low-level CPU cache visibility and control, rendering conventional cache optimization techniques ineffective. To address this, we propose CacheX—the first hardware- and hypervisor-agnostic, purely software-based fine-grained cache abstraction and probing framework for VMs. CacheX leverages eviction sets to accurately infer last-level cache (LLC) topology and contention states. It further introduces an LLC contention-aware task scheduler and a virtual-coloring–aware page cache management mechanism. The entire solution is fully integrated into the x86 Linux kernel. Evaluation across mainstream public cloud VMs demonstrates that CacheX improves average cache utilization by 32.7% and boosts performance of critical workloads by 14.2%–28.5%, effectively overcoming a fundamental bottleneck in cloud environment cache optimization.

Technology Category

• Machine Learning: Hardware-aware ML
• Computer Vision: Learning & Optimization for CV
• Search and Optimization: Distributed Search

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Virtualization and resource management in Web systems and infrastructures
• Economics, Online Markets and Human Computation: Architectures and workflows that use LLMs for crowd work
• Security and Privacy: Large-scale security measurements

Abstract

This paper shows that cache-based optimizations are often ineffective in cloud virtual machines (VMs) due to limited visibility into and control over provisioned caches. In public clouds, CPU caches can be partitioned or shared among VMs, but a VM is unaware of cache provisioning details. Moreover, a VM cannot influence cache usage via page placement policies, as memory-to-cache mappings are hidden. The paper proposes a novel solution, CacheX, which probes accurate and fine-grained cache abstraction within VMs using eviction sets without requiring hardware or hypervisor support, and showcases the utility of the probed information with two new techniques: LLC contention-aware task scheduling and virtual color-aware page cache management. Our evaluation of CacheX's implementation in x86 Linux kernel demonstrates that it can effectively improve cache utilization for various workloads in public cloud VMs.

Problem

• Optimizing CPU cache utilization in cloud VMs
• Addressing limited cache visibility and control in VMs
• Improving cache performance without hypervisor support

Innovation

• Probes cache abstraction using eviction sets
• Implements LLC contention-aware task scheduling
• Uses virtual color-aware page cache management