Scalable Approximate Algorithms for Optimal Transport Linear Models

2025-04-06

AI Summary

This work addresses generalized nonnegative linear regression with an entropically regularized optimal transport (OT) data-fitting term, incorporating arbitrary convex regularization on weights and composite convex losses between marginal distributions—applicable to spectral unmixing, music transcription, and related tasks. Methodologically, we propose the first unified, scalable approximate solver: a Sinkhorn-type iterative framework specifically designed for general convex structures, yielding concise, theoretically rigorous multiplicative update rules. The algorithm supports efficient parallel implementation, ensures convergence, and scales to large problems. Experimentally, our method significantly outperforms existing OT-based regression approaches in both accuracy and runtime, while markedly improving model generalization and practical applicability.

Technology Category

• Search and Optimization: Non-convex Optimization
• Machine Learning: Optimization
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• Systems and Infrastructure for Web, Mobile and WoT: Experiences and lessons learnt from Web-based algorithms and system deployments

Abstract

Recently, linear regression models incorporating an optimal transport (OT) loss have been explored for applications such as supervised unmixing of spectra, music transcription, and mass spectrometry. However, these task-specific approaches often do not generalize readily to a broader class of linear models. In this work, we propose a novel algorithmic framework for solving a general class of non-negative linear regression models with an entropy-regularized OT datafit term, based on Sinkhorn-like scaling iterations. Our framework accommodates convex penalty functions on the weights (e.g. squared-$ell_2$ and $ell_1$ norms), and admits additional convex loss terms between the transported marginal and target distribution (e.g. squared error or total variation). We derive simple multiplicative updates for common penalty and datafit terms. This method is suitable for large-scale problems due to its simplicity of implementation and straightforward parallelization.

Problem

• Generalizing linear regression with optimal transport loss
• Solving non-negative linear regression with OT datafit
• Enabling large-scale problems via scalable algorithms

Innovation

• Sinkhorn-like scaling for entropy-regularized OT
• Convex penalty functions on model weights
• Multiplicative updates for large-scale efficiency