QMDB: Quick Merkle Database

2025-01-09

AI Summary

Blockchain world-state storage suffers from poor SSD compatibility, low throughput, and high verifiability overhead. This paper proposes VDB, an SSD-optimized verifiable database, introducing the first append-only authenticated data structure that enables single-SSD reads, O(1) I/O update complexity, and lightweight in-memory Merklization. VDB integrates an SSD-aware storage engine, batched Merkle trees, DRAM-efficient compact node encoding, and incremental hashing. Experiments demonstrate a throughput of 2 million state updates per second—6× higher than RocksDB and 8× higher than state-of-the-art verifiable databases. A single server supports up to 15 billion entries (10× Ethereum’s 2024 state size), with a theoretical capacity of 280 billion entries.

Technology Category

• Data Mining & Knowledge Management: Representing, Reasoning, and Using Provenance, Trust
• Search and Optimization: Distributed Search
• Intelligent Robots: State Estimation

Application Category

• Security and Privacy: Blockchains and distributed ledgers
• Responsible Web: Data and user privacy-enhancing technologies for the Web
• Systems and Infrastructure for Web, Mobile and WoT: Data management and stream processing for Web, mobile and wireless applications

Abstract

Updating, managing, and proving world state are key bottlenecks facing the execution layer of blockchains today. Existing storage solutions are not flash-optimized and suffer from high flash write amplification and excessive DRAM requirements, forcing a trade-off between throughput and decentralization. We present the Quick Merkle Database (QMDB), an SSD-optimized authenticated data structure that delivers a superset of the features of existing databases. QMDB's append-only design enables 1 SSD read per state access, $O(1)$ I/Os for updates, and in-memory Merkleization on a DRAM footprint small enough to fit on consumer-grade PCs. We demonstrate that QMDB offers a significant leap in throughput ($6 imes$ over RocksDB and $8 imes$ over a state-of-the-art verifiable database) and validate its scalability on datasets up to 15 billion entries ($10 imes$ Ethereum's state size in 2024). Our projections indicate QMDB could store a theoretical maximum of 280 billion entries on a single machine, far exceeding current blockchain requirements. QMDB scales across both commodity and enterprise hardware, achieving up to 2 million state updates per second. QMDB sets a new benchmark for verifiable databases, alleviating today's storage bottlenecks, lowering barriers to blockchain participation, and unlocking new blockchain applications.

Problem

• Blockchain Technology
• Flash Storage Incompatibility
• Big Data Efficiency

Innovation

• Flash-Optimized Merkle Database
• High-Throughput Data Processing
• Blockchain Scalability Enhancement