Model-guided Fuzzing of Distributed Systems

2024-10-03

arXiv.org

AI Summary

Traditional randomized testing struggles to cover critical semantic states in distributed systems implementations. Method: This paper proposes the first coverage-guided fuzzing framework leveraging TLA+ abstract models. It defines semantics-aware coverage targets directly from protocol-level TLA+ specifications and integrates state-space-aware mutation strategies with a scheduling-aware execution engine to systematically explore protocol-critical states. Unlike conventional approaches relying solely on code or scheduling coverage, our method bridges the semantic gap by deeply embedding formal models into the fuzzing feedback loop. Results: Evaluation on Etcd-raft and RedisRaft demonstrates significantly improved coverage, accelerated bug discovery, and the identification of 13 previously unknown bugs—including four deep protocol logic flaws that are only triggerable under model-guided exploration.

Technology Category

Search and Optimization: Distributed SearchPlanning, Routing, and Scheduling: Model-Based ReasoningMachine Learning: Distributed Machine Learning & Federated Learning

Application Category

Search and Retrieval-Augmented AI: Web evaluation methodologies and metricsSemantics and Knowledge: Provenance, trust, security and privacy, and ethical issues in managing semantic dataSecurity and Privacy: Large-scale security measurements

Abstract

We present a coverage-guided testing algorithm for distributed systems implementations. Our main innovation is the use of an abstract formal model of the system that is used to define coverage. Such abstract models are frequently developed in early phases of protocol design and verification but are infrequently used at testing time. We show that guiding random test generation using model coverage can be effective in covering interesting points in the implementation state space. We have implemented a fuzzer for distributed system implementations and abstract models written in TLA+. Our algorithm shows better coverage over purely random exploration as well as random exploration guided by different notions of scheduler coverage and mutation. In particular, we show consistently higher coverage and detect bugs faster on implementations of distributed consensus protocols such as those in Etcd-raft and RedisRaft. Moreover, we discovered 13 previously unknown bugs in their implementations, four of which could only be detected by model-guided fuzzing.

Problem

• Guiding fuzzing using abstract models for distributed systems
• Improving test coverage over random exploration methods
• Detecting bugs in distributed consensus protocol implementations

Innovation

• Model-guided fuzzing with abstract formal models
• Coverage-guided testing using TLA+ specifications
• Random test generation driven by model coverage