Adjusted Kolmogorov Complexity of Binary Words with Empirical Entropy Normalization

2025-12-24

AI Summary

Binary-word Kolmogorov complexity is susceptible to bias from imbalanced symbol frequencies, hindering its ability to capture intrinsic descriptive complexity. Method: We propose entropy-normalized Kolmogorov complexity, which incorporates empirical entropy as a normalization factor—marking the first integration of entropy directly into the definition of Kolmogorov complexity to disentangle combinatorial effects. Contribution/Results: Theoretically, we establish a linear convergence relationship between this normalized complexity and Martin-Löf randomness under constructive exchangeable measures, proving that the normalized complexity converges almost surely to 1 under such measures. Furthermore, we construct counterexamples demonstrating the necessity of the measure’s regularity condition. This framework provides a novel tool for randomness testing and structural complexity analysis of binary sequences, combining theoretical rigor with computational feasibility.

Technology Category

• Machine Learning: Information Theory
• Knowledge Representation and Reasoning: Computational Complexity of Reasoning
• Data Mining & Knowledge Management: Data Compression

Application Category

• Security and Privacy: Large-scale security measurements
• Web Mining and Content Analysis: Normalization, clustering, classification, and summarization of Web text
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs

Abstract

Kolmogorov complexity of a finite binary word reflects both algorithmic structure and the empirical distribution of symbols appearing in the word. Words with symbol frequencies far from one half have smaller combinatorial richness and therefore appear less complex under the standard definition. In this paper an entropy-normalized complexity measure is introduced that divides the Kolmogorov complexity of a word by the empirical entropy of its observed distribution of zeros and ones. This adjustment isolates intrinsic descriptive complexity from the purely combinatorial effect of symbol imbalance. For Martin Löf random sequences under constructive exchangeable measures, the adjusted complexity grows linearly and converges to one. A pathological construction shows that regularity of the underlying measure is essential. The proposed framework connects Kolmogorov complexity, empirical entropy, and randomness in a natural manner and suggests applications in randomness testing and in the analysis of structured binary data.

Problem

• Adjusts Kolmogorov complexity to remove bias from symbol imbalance
• Introduces entropy-normalized measure to isolate intrinsic descriptive complexity
• Connects adjusted complexity, empirical entropy, and randomness for applications

Innovation

• Adjusted Kolmogorov complexity normalized by empirical entropy
• Isolates intrinsic descriptive complexity from symbol imbalance
• Connects Kolmogorov complexity, empirical entropy, and randomness