Hybrid Summary Statistics

2024-10-10

arXiv.org

AI Summary

Simulation-based inference (SBI) under sparse parameter sampling suffers from poor posterior estimation fidelity and robustness, especially in low-data regimes. Method: This paper proposes a mutual information-driven posterior estimation framework that jointly optimizes physics-informed prior statistics and neural network–based embedding representations via two complementary mutual information maximization objectives—marking the first approach to co-optimize these distinct statistical sources. Contribution/Results: The method significantly enhances information preservation and robustness of posterior estimates with limited simulations. Experiments on two cosmological datasets demonstrate superior extraction of non-Gaussian parameter dependencies compared to neural-only summary statistics or naive concatenation baselines. Inference stability under small-sample conditions is markedly improved. By integrating domain knowledge with deep representation learning through principled information-theoretic objectives, the approach establishes a novel, interpretable, and high-information statistical learning paradigm for sparse-simulation inference.

Technology Category

• Search and Optimization: Sampling/Simulation-based Search
• Reasoning under Uncertainty: Probabilistic Inference
• Machine Learning: Calibration & Uncertainty Quantification

Application Category

• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

We present a way to capture high-information posteriors from training sets that are sparsely sampled over the parameter space for robust simulation-based inference. In physical inference problems, we can often apply domain knowledge to define traditional summary statistics to capture some of the information in a dataset. We show that augmenting these statistics with neural network outputs to maximise the mutual information improves information extraction compared to neural summaries alone or their concatenation to existing summaries and makes inference robust in settings with low training data. We introduce 1) two loss formalisms to achieve this and 2) apply the technique to two different cosmological datasets to extract non-Gaussian parameter information.

Problem

• Extracting high-information posteriors from sparsely sampled training sets
• Improving information extraction by augmenting traditional statistics with neural networks
• Making simulation-based inference robust in low training data settings

Innovation

• Hybrid summary statistics combine neural networks with domain knowledge
• Two loss formalisms maximize mutual information for robust inference
• Technique extracts non-Gaussian information from cosmological datasets