VIREL: Verified Integer-Residual Encoding on Lattices for Exact and Error-Bounded Floating-Point Time-Series Compression

2026-07-24

AI Summary

This work addresses the limitations of existing floating-point time series compression methods, which operate directly on IEEE binary representations and overlook the fact that real-world data often originate from decimal or physically constrained resolution grids, thereby struggling to balance precision, error control, and compression efficiency. To overcome this, we propose VIREL, a novel framework that maps floating-point values onto a verification grid, performs residual prediction in the integer domain, and incorporates grid-step normalization and a multi-path routing mechanism. This design enables mixed-precision support, robust outlier handling, and streamable, page-based compression with exact or error-bounded guarantees. Integrated into Apache TsFile, VIREL achieves a 9.65× compression ratio for lossless scenarios and 12.11× under a 1e-3 error bound—reducing bit usage by 12.54%—while strictly adhering to per-point error constraints on standard benchmarks.

Abstract

Streaming floating-point data are often generated on decimal or physical resolution lattices, but most exact compressors still model the IEEE binary representation directly. This paper presents VIREL, a page-streamable framework for exact and error-bounded floating-point time-series compression. VIREL first admits each value into a verified decimal or error lattice, then predicts residuals in the resulting integer generation domain rather than in IEEE-word XOR space. To handle mixed precision and exceptions, VIREL routes values into compatible lanes and keeps predictor state local to each lane. Its compression-first exact profile further applies costed lattice-step normalization: for an affine sub-lattice q = d z + r, it encodes the compact coordinate z only when the complete frame cost decreases. The error-bounded profile applies the same idea with an exact divisibility rule, storing q/d only when it reconstructs the same lattice point and therefore consumes no error budget. Across canonical exact streams with independent 1,024-value pages, VIREL-Exact-Fast reaches 6.0243x compression, while VIREL-Exact-Upper reaches 7.0287x and emits substantially fewer bytes than the strongest evaluated prior exact codecs. On 74.70 million values in 48 independently paged streams, the two exact profiles reach 8.0629x and 9.6490x. At error bound 1e-3 on 15 Serf streams, VIREL-EB reaches 12.1094x, emits 12.54 percent fewer bits than the strongest compliant external baseline, and preserves all checked pointwise bounds. Ablations show that integer-domain residual prediction is the primary source of gain, while lattice-step normalization and route-persistent multi-lane prediction preserve that gain under mixed resolution. VIREL also scales across CPU cores and runs as an Apache TsFile page codec.

Problem

• floating-point compression
• time-series
• lattice encoding
• exact compression
• error-bounded compression

Innovation

• lattice-based compression
• integer-domain residual prediction
• error-bounded compression
• multi-lane routing
• costed lattice-step normalization