Tiara: A Programmable Line-Rate ISA for Remote Memory Access

2026-06-10

AI Summary

This work addresses the performance bottleneck caused by address dependencies in remote memory access—termed the “indirection wall”—which incurs multiple round-trip latencies and severely limits applications such as graph traversal, page table lookups, distributed locking, and disaggregated large model inference. The authors propose a compact, statically verifiable eBPF-like instruction set executable on memory-side RDMA NICs, enabling pre-registered programs to resolve indirect accesses locally at the remote node and collapse multi-hop dependencies into a single communication round. Leveraging an FPGA-based memory-side NIC architecture combined with optimized remote memory primitives, the system achieves line-rate, secure processing of complex indirect accesses. Experiments demonstrate 2.85× lower latency and 3.4× higher throughput for graph traversal, 62% reduced page table lookup latency, 2.9× lower latency for contention-free distributed locks, 2.8× higher throughput for disaggregated PagedAttention, and 1.88× lower latency for MoE expert gathering.

Abstract

RDMA one-sided verbs are the natural primitive for memory disaggregation, but they require the client to supply the exact remote address. The 1-RTT performance breaks down when the target address depends on data that must first be read from remote memory, a pattern we call the Indirection Wall. Indirection is pervasive: graph traversals follow pointers hop by hop, address translation walks multi-level page tables, distributed coordination requires conditional multi-host logic, and disaggregated LLM inference must resolve paged KV caches through block-table lookups. Each level of indirection costs one sequentially dependent network round-trip, yet offloading to existing RDMA NICs either consumes remote CPU cycles or has limited throughput. We present Tiara, a compact, statically verifiable instruction set that executes on the memory-side NIC. Tiara operators are pre-registered programs, analogous to eBPF programs in the kernel, that resolve indirection locally, collapsing multi-RTT dependent chains into a single round-trip. On an FPGA-based prototype, Tiara reduces 10-hop graph-traversal latency by 2.85x over one-sided RDMA while sustaining 3.4x higher throughput, cuts page-table walk latency by 62%, reduces uncontended distributed-lock latency by 2.9x, achieves 2.8x throughput for disaggregated PagedAttention at 8 KB blocks, and 1.88x MoE expert-gather latency at 32 experts.

Problem

• Indirection Wall
• Remote Memory Access
• RDMA
• Memory Disaggregation
• Network Latency

Innovation

• Tiara
• RDMA
• memory disaggregation
• indirection resolution
• programmable NIC