Unified Lookup-Table Inference with Signed-Digit K/V Caches for Ternary LLMs

2026-08-04

AI Summary

This work addresses the challenge of efficiently unifying attention mechanism’s key/value (K/V) caching and ternary projections in ternary large language models, which suffer from mismatches in precision and architecture. The authors propose a unified lookup-table-based inference method that stores K/V states in a scaled multi-plane signed-digit representation and processes them directly via activation-driven lookup tables, thereby avoiding explicit reconstruction of dense tensors. This approach enables, for the first time, unified table-driven execution of both linear projections and attention operations in ternary LLMs, supporting online K/V construction, bounded handling of incomplete value blocks, and shared multi-stream datapaths. Experiments demonstrate that the proposed method effectively balances cache overhead, model quality, and hardware efficiency on both natively trained and post-training quantized ternary models.

Abstract

Ternary LLMs make their weight-dominated projections compact and efficient, but attention remains a mismatch: its K/V cache is created online and is typically processed by a separate higher-precision engine. Compressing this cache alone does not resolve the mismatch. To execute attention with the same lookup-table machinery as ternary projections, values accumulated in one reduction must retain a compatible representation and scale. This requirement also differs for keys and values during causal decoding, because newly generated values may belong to an unfinished cache block. This work develops a unified lookup-table inference approach for ternary LLMs. It stores runtime K/V states as scaled multi-plane signed digits organized around the reduction structure of attention. The resulting digit planes are consumed directly by activation-derived tables, avoiding dense K/V materialization between cache storage and attention computation. The design combines online K/V formation, bounded handling of incomplete value blocks, and a shared multi-stream datapath for Linear projections and attention. A constraint-guided search selects the representation and execution policy for a target quality--efficiency trade-off. Experiments on native and post-training ternary models validate the approach across cache capacity, model quality, and hardware efficiency.

Problem

• Ternary LLMs
• K/V cache
• attention mechanism
• lookup-table inference
• causal decoding

Innovation

• ternary LLMs
• lookup-table inference
• signed-digit K/V caches
• unified attention-projection datapath
• multi-plane representation