infomeasure: a comprehensive Python package for information theory measures and estimators

2025-05-07

Scientific Reports

AI Summary

Information-theoretic analyses face practical limitations due to high computational complexity, fragmented software ecosystems, and poor reproducibility. To address these challenges, we introduce *infomeasure*, an open-source Python toolkit that—uniquely—provides a unified framework for computing fundamental information measures—including entropy, mutual information, transfer entropy, and divergence—for both discrete and continuous variables. The package integrates state-of-the-art estimators: kernel density estimation, k-nearest neighbors, and plug-in methods, and further supports statistical inference via local values, p-values, and t-scores. Accuracy is rigorously validated against analytical solutions. In application, *infomeasure* successfully detects nonlinear information flow in electroencephalographic (EEG) time series, demonstrating empirical utility. By standardizing implementation, documentation, and validation, the toolkit substantially lowers methodological barriers, enhances cross-study reproducibility, and improves transparency in information-theoretic analysis.

Technology Category

• Machine Learning: Information Theory
• Natural Language Processing: Information Extraction
• Cognitive Modeling & Cognitive Systems: Neural Spike Coding

Application Category

• Web Mining and Content Analysis: Web measurements
• Security and Privacy: Large-scale security measurements
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs

Abstract

Information theory, i.e. the mathematical analysis of information and of its processing, has become a tenet of modern science; yet, its use in real-world studies is usually hindered by its computational complexity, the lack of coherent software frameworks, and, as a consequence, low reproducibility. We here introduce infomeasure, an open-source Python package designed to provide robust tools for calculating a wide variety of information-theoretic measures, including entropies, mutual information, transfer entropy and divergences. It is designed for both discrete and continuous variables; implements state-of-the-art estimation techniques; and allows the calculation of local measure values, p-values and t-scores. By unifying these approaches under one consistent framework, infomeasure aims to mitigate common pitfalls, ensure reproducibility, and simplify the practical implementation of information-theoretic analyses. In this contribution, we explore the motivation and features of infomeasure; its validation, using known analytical solutions; and exemplify its utility in a case study involving the analysis of human brain time series.

Problem

• Address computational complexity in information theory applications
• Provide unified software for diverse information-theoretic measures
• Enhance reproducibility in information-theoretic analyses

Innovation

• Open-source Python package for information theory
• Supports discrete and continuous variables analysis
• Implements state-of-the-art estimation techniques