Technology Category
Machine Learning: Information TheoryIntelligent Robots: State EstimationReasoning under Uncertainty: Relational Probabilistic Models
FMMI: Flow Matching Mutual Information Estimation
🤖 AI Summary
Estimating mutual information (MI) in high-dimensional settings suffers from low efficiency, poor accuracy, and limited stability—especially when MI values span several orders of magnitude. To address this, we propose the first normalized flow-based MI estimator grounded in flow matching (FM). Unlike prevailing discriminative approaches, our method directly models the invertible density transformation between the joint and marginal distributions via normalizing flows, enabling end-to-end, differentiable MI estimation. By integrating flow matching into the MI estimation framework, we achieve both theoretical rigor—guaranteeing unbiased gradient estimation—and computational scalability. Experiments on multivariate benchmark tasks demonstrate that our estimator significantly outperforms baselines including InfoNCE and MINE in estimation accuracy, converges faster, incurs lower computational overhead, and exhibits strong robustness to both extremely small and large MI values.
Technology Category
Application Category
📝 Abstract
We introduce a novel Mutual Information (MI) estimator that fundamentally reframes the discriminative approach. Instead of training a classifier to discriminate between joint and marginal distributions, we learn a normalizing flow that transforms one into the other. This technique produces a computationally efficient and precise MI estimate that scales well to high dimensions and across a wide range of ground-truth MI values.
Problem
Research questions and friction points this paper is trying to address.
Estimating mutual information efficiently in high dimensions
Transforming distributions using normalizing flows instead of classifiers
Creating scalable mutual information estimation across various ground-truth values
Innovation
Methods, ideas, or system contributions that make the work stand out.
Uses normalizing flow to transform distributions
Estimates mutual information without discriminative classifier
Provides efficient high-dimensional scalable estimation
🔎 Similar Papers
No similar papers found.
