Representation Learning for Exact Preimages

2026-09-25

AI Summary

This study addresses the challenge that preimages of neural network predictive models are difficult to represent explicitly and computationally expensive to optimize inversely. To overcome this, we propose the TRIO framework, which introduces a novel preimage decomposition mechanism and nonlinear radial transformations to decouple forward representations from geometric preimages. By reformulating complex nonlinear mappings as simple geometric level sets, TRIO integrates neural representation learning with convex optimization theory for efficient solving. We establish a universal approximation theorem demonstrating that the framework can precisely approximate arbitrary continuous mappings and their non-convex preimages. Consequently, TRIO preserves model expressiveness while enabling efficient inversion and global optimization, significantly enhancing computational tractability.

Abstract

Modern neural predictors can model highly nonlinear maps, but many scientific and engineering tasks require reasoning in the opposite direction: given a performance or safety level, the goal is to characterize the preimage, that is, the complete set of inputs which meet the desired target level and optimize over that set. For expressive neural predictors, however, such preimages typically have no explicit representation and are expensive to recover or optimize over. This creates a fundamental three-way challenge between expressive forward prediction, accurate preimage approximation, and tractable optimization over the preimage for downstream tasks. We introduce TRIO (tractable representations for preimage learning and inverse optimization), a framework for learning representations that make these objectives compatible by construction. Our key contribution is a preimage factorization: the forward model remains expressive through nonlinear radial transformations (including neural networks), while, under inversion, each transformation reduces to a single scalar radius, which yields simple geometric level sets. This yields an explicit geometric representation that is reusable for downstream optimization over the preimage, and, for linear objectives, we show that this admits a closed-form global solution. We finally prove a universal approximation theorem which shows that TRIO can approximate any continuous forward map and its entire family of potentially disconnected, nonconvex preimages arbitrarily well. Hence, TRIO combines expressive forward modeling, exact preimage recovery, and tractable global downstream optimization over preimages by design.

Problem

• Representation Learning
• Preimage
• Inverse Optimization
• Neural Predictors
• Tractable Optimization

Innovation

• Representation Learning
• Preimage Factorization
• Inverse Optimization
• Universal Approximation
• Radial Transformations