Smoothed Distance Kernels for MMDs and Applications in Wasserstein Gradient Flows

2025-04-10

AI Summary

This work addresses the non-differentiability of negative distance kernels—e.g., $k(x,y) = -|x-y|$—at $x=y$, which precludes their use in rigorous Wasserstein gradient flow analysis. To resolve this, we construct a novel class of Lipschitz-smooth kernels that are conditionally positive definite, exhibit near-linear growth, and admit a sliceable structure. Our construction uniquely integrates smoothed one-dimensional absolute value functions with Riemann–Liouville fractional integration, achieving both theoretical completeness and numerical practicality. The proposed kernel strictly satisfies the existence and convergence conditions for Wasserstein gradient flows. Empirically, in sliced maximum mean discrepancy (MMD) gradient descent experiments, it matches the performance of the original negative distance kernel. This is the first theoretically grounded construction that simultaneously preserves the favorable statistical properties of distance-based kernels—including unbiased estimation and characteristicness—and guarantees smoothness required for gradient-based optimization and PDE analysis.

Technology Category

• Machine Learning: Kernel Methods
• Search and Optimization: Non-convex Optimization
• Natural Language Processing: Learning & Optimization for NLP

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Systems and Infrastructure for Web, Mobile and WoT: Data management and stream processing for Web, mobile and wireless applications
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Negative distance kernels $K(x,y) := - |x-y|$ were used in the definition of maximum mean discrepancies (MMDs) in statistics and lead to favorable numerical results in various applications. In particular, so-called slicing techniques for handling high-dimensional kernel summations profit from the simple parameter-free structure of the distance kernel. However, due to its non-smoothness in $x=y$, most of the classical theoretical results, e.g. on Wasserstein gradient flows of the corresponding MMD functional do not longer hold true. In this paper, we propose a new kernel which keeps the favorable properties of the negative distance kernel as being conditionally positive definite of order one with a nearly linear increase towards infinity and a simple slicing structure, but is Lipschitz differentiable now. Our construction is based on a simple 1D smoothing procedure of the absolute value function followed by a Riemann-Liouville fractional integral transform. Numerical results demonstrate that the new kernel performs similarly well as the negative distance kernel in gradient descent methods, but now with theoretical guarantees.

Problem

• Non-smoothness of negative distance kernels in MMDs
• Lack of theoretical guarantees for Wasserstein gradient flows
• Need for Lipschitz differentiable kernel with similar properties

Innovation

• Smoothed 1D kernel for MMDs
• Lipschitz differentiable distance kernel
• Riemann-Liouville fractional integral transform