The Geometry of Anisotropic Dilation for Optimal Regularization

2026-10-06

AI Summary

This study addresses the challenge of adapting regularizers to data geometry in data-driven inverse problems by proposing an anisotropic dilation transform. This method adjusts radial statistics via a direction-preserving mapping, enabling optimal data adaptation and joint learning under a fixed base regularizer. Theoretically, leveraging the transitivity of the transform in radial statistical space, we derive a closed-form solution for optimal adaptation, prove its superiority over isotropic scaling, and establish finite-sample generalization bounds. Technically, the approach integrates Gibbs-type regularization, variational methods, and sparse ReLU network parameterization. Experiments demonstrate that the proposed method significantly enhances inverse problem performance in controlled two-dimensional settings and MNIST denoising tasks.

Abstract

A central question in data-driven inverse problems is how to construct a regularizer that adapts to the geometry of the data distribution. Recent work in optimal regularization shows that, within a broad Gibbs class, the regularizer best matched to a distribution $P$ is determined by a single, direction-dependent radial summary statistic $ρ_P$. This suggests a way to control regularizer geometry by transforming the data. In particular, which transformations act on $ρ_P$ in a simple, explicit way? Can they improve the optimization properties of the resulting variational problems or adapt a fixed base regularizer to data? To address these questions, we introduce anisotropic dilation, a direction-preserving map that rescales each point along its Euclidean ray by a positive profile on the sphere. Despite its simplicity, this family acts transitively on the space of radial summary statistics: any target regularizer in the Gibbs class can be reached from any source distribution by a single explicit profile. We characterize the resulting orbit structure on distributions, derive in closed form the profile that optimally adapts a fixed base regularizer to the data, and show that its improvement over isotropic rescaling is governed by a Jensen gap that is provably positive for several natural families. We also establish finite-sample generalization bounds for jointly learning the base regularizer and anisotropic profile, with explicit rates when the logarithm of the profile is parameterized by linear feature models or sparse ReLU networks. Building on this theory, we parameterize the base regularizer and anisotropic profile and learn them jointly from samples, yielding a data-adaptive regularizer for variational inverse problems. Experiments on a controlled two-dimensional family and MNIST denoising show that learning the profile further improves performance.

Problem

• inverse problems
• optimal regularization
• anisotropic dilation
• data-driven regularizer
• radial summary statistic

Innovation

• Anisotropic Dilation
• Optimal Regularization
• Radial Summary Statistic
• Variational Inverse Problems
• Finite-Sample Generalization Bounds