Lossless Compression via Chained Lightweight Neural Predictors with Information Inheritance

2026-04-16

AI Summary

This work addresses the longstanding trade-off between compression ratio and encoding/decoding efficiency in lossless compression by proposing a novel approach based on a chained lightweight neural predictor. The method dynamically selects the smallest neural network unit according to the statistical characteristics of the input data for probability estimation and incorporates an information inheritance mechanism to enhance compression efficiency. Its core innovations include a pioneering chained lightweight neural prediction architecture and an efficient GPU implementation. Experimental results demonstrate that the proposed method achieves compression ratios comparable to current state-of-the-art techniques such as PAC, while significantly improving throughput: encoding speed increases by 1.2–6.3× and decoding speed by 2.8–12.3× on consumer-grade GPUs.

Technology Category

Data Mining & Knowledge Management: Data CompressionMachine Learning: Learning on the Edge & Model CompressionCognitive Modeling & Cognitive Systems: Neural Spike Coding

Application Category

Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphsEconomics, Online Markets and Human Computation: Cost models of using LLMs in production systemsWeb Mining and Content Analysis: Large pretrained models with web data

Abstract

This paper is dedicated to lossless data compression with probability estimation using neural networks. First, we propose a probability estimation architecture based on a chain of neural predictors, so that each unit of the chain is defined as a neural network with the minimum possible number of weights, which is sufficient for efficient compression of data generated by Markov sources of a given order. We show that this architecture allows us to minimize the overall number of weights participating in the probability estimation process depending on the statistical properties of the input data. Second, in order to improve compression efficiency, we introduce an information inheritance mechanism, where the probability estimate obtained by a low-order unit is used at the next higher-order unit. Experimental results show that the proposed lossless data compressor equipped with the chained probability estimation architecture provides compression ratios close to the state-of-the-art PAC compressor. At the same time, it outperforms PAC by a factor of 1.2 to 6.3 in encoding throughput and by a factor of 2.8 to 12.3 in decoding throughput on a consumer GPU.

Problem

• lossless compression
• probability estimation
• neural predictors
• Markov sources
• throughput

Innovation

• chained neural predictors
• information inheritance
• lossless compression
• probability estimation
• lightweight neural networks