Technology Category
Data Mining & Knowledge Management: Data CompressionMachine Learning: Learning on the Edge & Model CompressionComputer Vision: Diffusion Models for Vision
Floating-Point Data Transformation for Lossless Compression
🤖 AI Summary
Existing lossless compression methods for floating-point data—such as medical images and large model weights—suffer from low compression ratios due to their sensitivity to numerical precision and reliance on generic byte-stream compression or field-agnostic approaches. To address this, we propose Typed Data Transformation (DTT), a semantic-aware preprocessing technique that decouples the sign, exponent, and mantissa fields of floating-point numbers and reorganizes them across samples to explicitly exploit byte-level structural correlations. DTT is designed to synergize with general-purpose compressors (e.g., zstd) and enables end-to-end, high-throughput lossless compression on both CPU and GPU platforms. Experiments across diverse floating-point datasets demonstrate that DTT achieves an average 1.16× improvement in compression ratio and 1.18–3.79× higher compression/decompression throughput over baselines, while guaranteeing strict losslessness and cross-platform compatibility.
Technology Category
Application Category
📝 Abstract
Floating-point data is widely used across various domains. Depending on the required precision, each floating-point value can occupy several bytes. Lossless storage of this information is crucial due to its critical accuracy, as seen in applications such as medical imaging and language model weights. In these cases, data size is often significant, making lossless compression essential. Previous approaches either treat this data as raw byte streams for compression or fail to leverage all patterns within the dataset. However, because multiple bytes represent a single value and due to inherent patterns in floating-point representations, some of these bytes are correlated. To leverage this property, we propose a novel data transformation method called Typed Data Transformation (DTT{}) that groups related bytes together to improve compression. We implemented and tested our approach on various datasets across both CPU and GPU. DTT{} achieves a geometric mean compression ratio improvement of 1.16$ imes$ over state-of-the-art compression tools such as zstd, while also improving both compression and decompression throughput by 1.18--3.79$ imes$.
Problem
Research questions and friction points this paper is trying to address.
Enhance lossless compression of floating-point data
Exploit inherent patterns in floating-point representations
Improve compression ratio and throughput simultaneously
Innovation
Methods, ideas, or system contributions that make the work stand out.
Groups related floating-point bytes together
Improves compression ratio and throughput
Works on both CPU and GPU platforms
🔎 Similar Papers
No similar papers found.
