Transparent and Efficient Live Migration across Heterogeneous Hosts with Wharf

2024-10-21

arXiv.org

AI Summary

LLM agents face significant challenges in cross-heterogeneous-host migration (e.g., x86/ARM, Linux/FreeBSD), including data confidentiality, low-latency responsiveness, high availability, and output integrity. Method: This paper proposes Vessel, a lightweight WebAssembly-based containerization framework. It introduces WASI as a unified abstraction for process state and OS interfaces, and designs the Dock runtime to enable dynamic migration-safety-point detection, cross-OS coordination, and latency-aware triggering—achieving transparent, real-time migration without code modification or service restart. Contribution/Results: Experiments demonstrate a 57% reduction in migration pause time while ensuring sensitive data remains within trusted boundaries and critical outputs are integrity-protected. Vessel natively supports C/Rust binaries, enabling load balancing, hot updates, and fault tolerance. It establishes a novel paradigm for trustworthy AI agent deployment across diverse infrastructure.

Technology Category

Multiagent Systems: Adversarial AgentsMachine Learning: Large Multimodal Models (LMMs)Cognitive Modeling & Cognitive Systems: Agent Architectures

Application Category

Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applicationsResponsible Web: Human-perceived consequences of algorithmic deployment on the webGraph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Live migration allows a user to move a running application from one machine (a source) to another (a destination) without restarting it. The technique has proven useful for diverse tasks including load balancing, managing system updates, improving data locality, and improving system resilience. Unfortunately, current live migration solutions fail to meet today's computing needs. First, most techniques do not support heterogeneous source and destination hosts, as they require the two machines to have the same instruction set architecture (ISA) or use the same operating system (OS), which hampers numerous live migration usecases. Second, many techniques are not transparent, as they require that applications be written in a specific high-level language or call specific library functions, which imposes barriers to entry for many users. We present a new lightweight abstraction, called a vessel, that supports transparent heterogeneous live migration. A vessel maintains a machine-independent encoding of a process's state, using WebAssembly abstractions, allowing it to be executed on nearly-arbitrary ISAs. A vessel virtualizes all of its OS state, using the WebAssembly System Interface (WASI), allowing it to execute on nearly arbitrary OS. We introduce docks and software systems that execute and migrate vessels. Docks face two key challenges: First, maintaining a machine-independent encoding at all points in a process is extremely expensive. So, docks instead ensure that a vessel is guaranteed to eventually reach a machine-independent point and delay the initiation of vessel migration until the vessel reaches such a point. Second, a dock may receive a vessel migration that originates from a dock executing on a different OS.

Problem

• Securely executing and migrating AI agents across heterogeneous environments
• Protecting sensitive user data while maintaining availability during failures
• Minimizing response latency and ensuring output safety for critical applications

Innovation

• WebAssembly-based secure container framework
• Two-way sandboxing with hardware enclaves
• Cross-platform migration using WebAssembly WASI