Phase-space entropy at acquisition reflects downstream learnability

2025-12-22

AI Summary

This paper addresses the challenge of quantifying information loss in downstream tasks during cross-modal data acquisition. We propose instrument-resolved phase-space entropy ΔS_ℬ—a domain-agnostic, modality-agnostic, and training-free metric for information preservation. Grounded in physical modeling of acquisition systems and coherence theory for periodic sampling, ΔS_ℬ establishes, for the first time, a universal theoretical link between phase-space entropy and downstream learnability (e.g., reconstruction or classification difficulty). Empirical validation across multimodal tasks—including image masked classification, accelerated MRI, and large-scale MIMO over-the-air measurements—demonstrates that the absolute value of ΔS_ℬ accurately ranks sampling strategies; zero-shot variable-density MRI masks selected via ΔS_ℬ match the performance of reconstruction-optimized, pre-trained alternatives; and ΔS_ℬ enables training-free, cross-task prediction of downstream learning difficulty.

Technology Category

• Machine Learning: Multimodal Learning
• Constraint Satisfaction and Optimization: Constraint Learning and Acquisition
• Computer Vision: Multi-modal Vision

Application Category

• Security and Privacy: Large-scale security measurements
• User Modeling, Personalization and Recommendation: User privacy protection in personalized systems
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG

Abstract

Modern learning systems work with data that vary widely across domains, but they all ultimately depend on how much structure is already present in the measurements before any model is trained. This raises a basic question: is there a general, modality-agnostic way to quantify how acquisition itself preserves or destroys the information that downstream learners could use? Here we propose an acquisition-level scalar $ΔS_{mathcal B}$ based on instrument-resolved phase space. Unlike pixelwise distortion or purely spectral errors that often saturate under aggressive undersampling, $ΔS_{mathcal B}$ directly quantifies how acquisition mixes or removes joint space--frequency structure at the instrument scale. We show theoretically that (ΔS_{mathcal B}) correctly identifies the phase-space coherence of periodic sampling as the physical source of aliasing, recovering classical sampling-theorem consequences. Empirically, across masked image classification, accelerated MRI, and massive MIMO (including over-the-air measurements), $|ΔS_{mathcal B}|$ consistently ranks sampling geometries and predicts downstream reconstruction/recognition difficulty emph{without training}. In particular, minimizing $|ΔS_{mathcal B}|$ enables zero-training selection of variable-density MRI mask parameters that matches designs tuned by conventional pre-reconstruction criteria. These results suggest that phase-space entropy at acquisition reflects downstream learnability, enabling pre-training selection of candidate sampling policies and as a shared notion of information preservation across modalities.

Problem

• Quantify how acquisition preserves or destroys information for downstream learners
• Measure phase-space entropy to rank sampling geometries without training
• Enable pre-training selection of sampling policies across different modalities

Innovation

• Phase-space entropy quantifies acquisition information preservation
• Instrument-resolved phase space measures joint space-frequency structure
• Zero-training selection of sampling policies across diverse modalities