Coupled Input-Output Dimension Reduction: Application to Goal-oriented Bayesian Experimental Design and Global Sensitivity Analysis

2024-06-19

arXiv.org

AI Summary

This paper addresses the challenge of jointly compressing high-dimensional input and output spaces in goal-oriented applications—such as sensor placement and sensitivity analysis—where conventional dimensionality reduction methods treat inputs and outputs independently. We propose an input–output co-dimensional reduction framework that jointly optimizes coupled input and output subspaces. Crucially, it reformulates the NP-hard combinatorial selection problem into a differentiable optimization over diagonal entries of a diagnostic matrix, obviating costly evaluations of the objective function. By integrating gradient-based upper-bound optimization, expected information gain, Sobol’ sensitivity indices, and spectral analysis of the diagnostic matrix, our method achieves substantial computational efficiency gains. Experiments demonstrate its effectiveness and scalability in sensor layout optimization and parameter importance ranking. The approach establishes a novel paradigm for high-dimensional, goal-oriented experimental design and global sensitivity analysis.

Technology Category

Search and Optimization: Combinatorial OptimizationMachine Learning: Dimensionality Reduction/Feature SelectionConstraint Satisfaction and Optimization: Constraint Optimization

Application Category

Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphsSearch and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for rankingWeb Mining and Content Analysis: Robustness and generalizability of Web mining methods

Abstract

We introduce a new method to jointly reduce the dimension of the input and output space of a function between high-dimensional spaces. Choosing a reduced input subspace influences which output subspace is relevant and vice versa. Conventional methods focus on reducing either the input or output space, even though both are often reduced simultaneously in practice. Our coupled approach naturally supports goal-oriented dimension reduction, where either an input or output quantity of interest is prescribed. We consider, in particular, goal-oriented sensor placement and goal-oriented sensitivity analysis, which can be viewed as dimension reduction where the most important output or, respectively, input components are chosen. Both applications present difficult combinatorial optimization problems with expensive objectives such as the expected information gain and Sobol' indices. By optimizing gradient-based bounds, we can determine the most informative sensors and most influential parameters as the largest diagonal entries of some diagnostic matrices, thus bypassing the combinatorial optimization and objective evaluation.

Problem

• Jointly reduce input and output space dimensions
• Optimize goal-oriented sensor placement and sensitivity analysis
• Bypass combinatorial optimization using gradient-based bounds

Innovation

• Jointly reduces input and output dimensions
• Optimizes gradient-based bounds for efficiency
• Identifies key sensors and parameters diagonally