Fast and Efficient Memory Reclamation For Serverless MicroVMs

2024-11-19

arXiv.org

AI Summary

In Function-as-a-Service (FaaS) environments, microVM-based serverless runtimes suffer from high latency and substantial bandwidth overhead during virtual memory hot-plug operations, compounded by the Linux kernel’s lack of explicit awareness and management of hot-plugged memory. Method: This paper proposes HotMem—a novel mechanism enabling explicit isolation and fine-grained lifecycle management of hot-plugged memory regions. It customizes the Linux v6.6 kernel to integrate memory region isolation, lightweight lifecycle tracking, and optimized, targeted memory reclamation paths. Contribution/Results: HotMem overcomes fundamental bottlenecks in conventional memory management: it accelerates memory reclamation by 10× over the state-of-the-art; achieves P99 function latency comparable to static over-provisioning models; and significantly improves microVM memory elasticity and overall resource utilization in FaaS deployments.

Technology Category

• Machine Learning: Hardware-aware ML
• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval
• Search and Optimization: Distributed Search

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Virtualization and resource management in Web systems and infrastructures
• Search and Retrieval-Augmented AI: Vertical and domain-specific search
• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization

Abstract

Resource elasticity is one of the key defining characteristics of the Function-as-a-Service (FaaS) serverless computing paradigm. In order to provide strong multi-tenant isolation, FaaS providers commonly sandbox functions inside virtual machines (VMs or microVMs). While compute resources assigned to VM-sandboxed functions can be seamlessly adjusted on the fly, memory elasticity remains challenging, especially when scaling down. State-of-the-art mechanisms for VM memory elasticity suffer from increased reclaim latency when memory needs to be released, compounded by CPU and memory bandwidth overheads. We identify the obliviousness of the Linux memory manager to the virtually hotplugged memory as the key issue hindering hot-unplug performance, and design HotMem, a novel approach for fast and efficient VM memory hot(un)plug, targeting VM-sandboxed serverless functions. Our key insight is that by segregating virtually hotplugged memory regions from regular VM memory, we are able to bound the lifetimes of allocations within these regions thus enabling their fast and efficient reclamation. We implement HotMem in Linux v6.6 and our evaluation shows that it is an order of magnitude faster than state-of-practice to reclaim VM memory, while achieving the same P99 function latency with a model that statically over-provisions VMs.

Problem

• Addresses slow VM memory reclamation in serverless functions
• Improves hot-unplug performance by segregating hotplugged memory
• Enables sub-second reclamation of multiple GiBs under FaaS load

Innovation

• Segregates hotplugged memory from regular VM memory
• Bounds allocation lifetime for fast memory reclamation
• Extends OS memory manager to enable sub-second reclamation