AsyncFS: Metadata Updates Made Asynchronous for Distributed Filesystems with In-Network Coordination

2024-10-11

arXiv.org

AI Summary

Distributed file systems suffer from high latency, directory contention, and load imbalance due to synchronous metadata updates—especially under dynamic or skewed workloads. This paper proposes the first P4-programmable switch–enabled asynchronous metadata update mechanism: directory updates are deferred until read time, while the switch lightweightly tracks state and aggregates batched write operations—achieving low overhead without compromising strong consistency. The key innovation lies in the co-design of in-network state awareness and asynchronous semantics. Evaluation shows that under skewed workloads, throughput improves by 13.34× and latency decreases by 61.6%. Under realistic workloads, end-to-end throughput increases by 21.1× over Ceph, 1.1× over IndexFS, and 30.1% over CFS-KV.

Technology Category

• Search and Optimization: Distributed Search
• Data Mining & Knowledge Management: Scalability, Parallel & Distributed Systems
• Multiagent Systems: Mechanism Design

Application Category

• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems
• Systems and Infrastructure for Web, Mobile and WoT: Web performance, measurement, and characterization
• Security and Privacy: Data transparency and provenance

Abstract

Distributed filesystems typically employ synchronous metadata updates, facing inherent challenges for access efficiency, load balancing, and directory contention, especially under dynamic and skewed workloads. This paper argues that synchronous updates are overly conservative for distributed filesystems. We propose AsyncFS with asynchronous metadata updates, allowing operations to return early and defer directory updates until respective read to enable latency hiding and conflict resolution. The key challenge is efficiently maintaining the synchronous semantics of metadata updates. To address this, AsyncFS is co-designed with a programmable switch, leveraging the constrained on-switch resources to holistically track directory states in the network with negligible cost. This allows AsyncFS to timely aggregate and efficiently apply delayed updates using batching and consolidation before directory reads. Evaluation shows that AsyncFS achieves up to 13.34$ imes$ and 3.85$ imes$ higher throughput, and 61.6% and 57.3% lower latency than two state-of-the-art distributed filesystems, InfiniFS and CFS-KV, respectively, on skewed workloads. For real-world workloads, AsyncFS improves end-to-end throughput by 21.1$ imes$, 1.1$ imes$ and 30.1% over Ceph, IndexFS and CFS-KV, respectively.

Problem

• Asynchronous metadata updates for distributed filesystems to improve efficiency
• Maintaining POSIX semantics while allowing early operation returns
• Using programmable switches to coordinate and batch directory updates

Innovation

• Asynchronous metadata updates for early operation returns
• Programmable switch co-design for tracking directory states
• Batching and consolidation of delayed updates before reads