Supervised Distributional Reduction via Optimal Transport and Dependence Maximization

2026-05-26

AI Summary

This work addresses the challenge of preserving both intrinsic geometric structure and task-relevant supervisory information in data compression. It proposes Supervised Distribution Reduction (SDR), a novel method that uniquely integrates optimal transport with dependence maximization by combining the Fused Gromov–Wasserstein distance and the Hilbert–Schmidt Independence Criterion (HSIC). This integration yields a target-aware, non-stationary geometric representation that simultaneously performs clustering, dimensionality reduction, and supervision preservation. By doing so, SDR enhances predictive performance while maintaining geometric fidelity. Furthermore, it introduces a new paradigm for adaptive kernel design in Gaussian processes, enabling kernels to dynamically respond to local variations in both data geometry and label structure.

Abstract

Learning representations that capture both intrinsic data geometry and target-relevant structure remains a fundamental challenge, particularly in settings where data reduction must balance compression with predictive fidelity. While distributional reduction-encompassing joint clustering and dimensionality reduction-offers a principled way to summarize data, its supervised variants remain relatively under-explored, despite the importance of retaining task-relevant signal for downstream prediction and decision-making. We propose Supervised Distributional Reduction (SDR), an algorithm for learning target-aware representations by combining optimal transport with explicit dependence maximization. SDR builds on the Fused Gromov-Wasserstein (FGW) objective to align the relational structure of the input distribution with a set of representative points, while augmenting it with a direct dependence term that encourages the learned embeddings to capture predictive signal more explicitly. This results in compact representations that reflect both geometric structure and supervision. Beyond representation learning, SDR naturally induces a data-dependent, non-stationary geometry that can be leveraged for settings such as Gaussian Process (GP) modelling. By redefining distances through target-aware distributional alignment, SDR enables the construction of adaptive kernels that respond to local variations in both data geometry and supervision, offering an optimal transport-based perspective on non-stationary kernel design.

Problem

• supervised distributional reduction
• optimal transport
• dependence maximization
• representation learning
• non-stationary kernels

Innovation

• Supervised Distributional Reduction
• Optimal Transport
• Dependence Maximization
• Fused Gromov-Wasserstein
• Non-stationary Kernels