Capacity Approximations for Insertion Channels with Small Insertion Probabilities

2024-11-22

arXiv.org

AI Summary

This work investigates the Shannon capacity of binary insertion channels under low insertion probability $p$, considering both the random-insertion model (where each bit is followed by an independent random bit with probability $p$) and Gallager’s model (where each bit is replaced by two independent random bits with probability $p$). Using Bernoulli$(1/2)$ input achievability, reverse bounds for stationary ergodic processes, coupling arguments, and asymptotic series expansions, we derive the first- and second-order asymptotic expansions of the capacity for both models. We establish that the capacity of each channel is of the form $1 - Theta(p)$, but with markedly distinct leading constants. Crucially, we prove rigorously that the random-insertion channel’s capacity strictly exceeds that of Gallager’s model by a gap of $Theta(p)$. This is the first fine-grained asymptotic characterization of capacity for synchronization-error channels, revealing a fundamental structural distinction between insertion and deletion channels in their capacity behavior.

Technology Category

• Machine Learning: Information Theory
• Cognitive Modeling & Cognitive Systems: Neural Spike Coding
• Planning, Routing, and Scheduling: Scheduling under Uncertainty

Application Category

• Security and Privacy: Large-scale security measurements
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Web Mining and Content Analysis: Models for Web evolution

Abstract

Channels with synchronization errors, exhibiting deletion and insertion errors, find practical applications in DNA storage, data reconstruction, and various other domains. Presence of insertions and deletions render the channel with memory, complicating capacity analysis. For instance, despite the formulation of an independent and identically distributed (i.i.d.) deletion channel more than fifty years ago, and proof that the channel is information stable, hence its Shannon capacity exists, calculation of the capacity remained elusive. However, a relatively recent result establishes the capacity of the deletion channel in the asymptotic regime of small deletion probabilities by computing the dominant terms of the capacity expansion. This paper extends that result to binary insertion channels, determining the dominant terms of the channel capacity for small insertion probabilities and establishing capacity in this asymptotic regime. Specifically, we consider two i.i.d. insertion channel models: insertion channel with possible random bit insertions after every transmitted bit and the Gallager insertion model, for which a bit is replaced by two random bits with a certain probability. To prove our results, we build on methods used for the deletion channel, employing Bernoulli(1/2) inputs for achievability and coupling this with a converse using stationary and ergodic processes as inputs, and show that the channel capacity differs only in the higher order terms from the achievable rates with i.i.d. inputs. The results, for instance, show that the capacity of the random insertion channel is higher than that of the Gallager insertion channel, and quantifies the difference in the asymptotic regime.

Problem

• Determining capacity of binary insertion channels with small insertion probabilities
• Comparing capacity between random insertion and Gallager insertion channel models
• Extending deletion channel methods to analyze insertion channel capacity asymptotics

Innovation

• Extends deletion channel methods to insertion channels
• Uses Bernoulli inputs for achievable rates
• Compares capacities of two insertion channel models