design dice loss variants

This work addresses the performance bottlenecks in medical image segmentation caused by data imbalance and the difficulty of detecting small lesions. To this end, the authors propose a novel polynomial Dice loss function based on Taylor expansion. This approach reformulates the conventional Dice loss into a polynomial form with adjustable higher-order terms, enabling flexible control over each term’s contribution to the overall loss and thereby refining the geometric properties and optimization dynamics of the loss function. Experimental results across multiple medical image segmentation tasks demonstrate that the proposed method significantly outperforms standard Dice loss and its Tversky variant, achieving superior segmentation accuracy and robustness.

This study addresses the problem of reducing the oracle complexity of gradient descent by leveraging the smoothness of the empirical risk minimization loss function over the data domain. To this end, the authors propose an inexact gradient method, termed PPI-GD, which approximates the full gradient via piecewise tensor-product polynomial interpolation constructed from isometric samples in the data domain, thereby significantly decreasing the number of gradient queries. This work is the first to integrate data-domain smoothness with interpolation techniques for optimizing oracle complexity and extends bicubic spline error analysis to the multivariate setting. Theoretical results demonstrate that, when the loss function is sufficiently smooth and the data dimension scales polylogarithmically with the sample size, PPI-GD outperforms existing gradient descent variants in both strongly convex and non-convex regimes.

This work investigates the generalization performance of random feature models under high-dimensional Gaussian data, focusing on how prediction accuracy depends on the number of random features $N$, the sample size $P$, and the input dimensionality $D$. Departing from conventional proportional scaling assumptions, we establish a rigorous mapping between random feature models and equivalent polynomial kernels. For the first time, we derive universal generalization error curves in non-asymptotic, finite-dimensional regimes. Our theoretical framework integrates statistical mechanics, random matrix theory, and high-dimensional asymptotic analysis, augmented by polynomial expansions and kernel equivalence modeling. We show that when $N$ and $P$ scale as power laws in $D$, the average generalization error is precisely characterized. Theory and numerical experiments agree quantitatively across several orders of magnitude in $N$ and $P$, and remain accurate even for finite $D$ and nonzero ratios $P/D^K$ or $N/D^L$, uncovering a fundamental equivalence between random features and polynomial kernels.

To address the challenge of efficiently and accurately approximating high-dimensional smooth and nonsmooth functions as well as partial differential equations (PDEs), this paper proposes Polynomial-Augmented Neural Networks (PANNs), which seamlessly integrate deep neural networks with orthogonal polynomial approximation. Our method introduces (1) a weak orthogonality constraint enforcing mutual orthogonality between network hidden layers and polynomial bases, and (2) a joint framework combining basis pruning and polynomial preconditioning to enhance numerical stability, accuracy, and scalability to high dimensions. Experiments demonstrate that PANNs consistently outperform pure deep neural networks and traditional polynomial methods on both function regression and PDE solving tasks. Notably, they achieve substantial accuracy gains for low-regularity functions, while exhibiting rapid convergence and strong generalization capability.

This work investigates the implicit bias of gradient descent in binary and multiclass classification under high-dimensional overparameterization, for arbitrary convex loss functions. We develop a unified analytical framework that, for the first time, extends asymptotic equivalence of implicit bias beyond exponential-tailed losses to general convex losses—bypassing the SVM intermediate representation and directly establishing asymptotic equivalence between gradient descent trajectories and the minimum-norm interpolating solution. Building upon Ji & Telgarsky (2021)’s dual analysis, we integrate high-dimensional asymptotics with convex optimization sensitivity theory to rigorously derive closed-form approximate solutions across diverse loss families. Our analysis recovers existing exact equivalence results and, crucially, quantifies for the first time the systematic deviation induced by out-of-distribution (OOD)-oriented losses on the interpolating solution.

This work addresses the challenges in multivariate polynomial models arising from the exponential growth of coefficient tensors with model order, the limited expressivity of existing tensor decomposition methods, and their sensitivity to feature ordering. To overcome these limitations, the authors propose an efficient optimization framework based on matrix product operators (MPOs). The approach leverages learnable MPO-based feature embeddings and compact polynomial weight tensors to achieve permutation-invariant function approximation. Furthermore, structured operators—such as projection, convolution, and masking—are incorporated to explicitly model weight symmetries, thereby transcending the expressivity bottlenecks of conventional tensor decompositions. Experimental results demonstrate that the proposed framework significantly outperforms existing tensor-decomposition-based polynomial models on both regression and classification benchmarks, offering a highly expressive, flexible, and computationally efficient solution for polynomial approximation.

This work addresses the problem of efficiently learning low-degree polynomial functions defined on the high-dimensional unit sphere, achieving near-minimax optimal sample complexity and regression risk without prior knowledge of the true degree. The authors propose an over-parameterized two-layer ReLU neural network equipped with enhanced feature representations and trained via projected gradient descent (GDP), enabling effective feature learning beyond the neural tangent kernel (NTK) regime. Their key contributions include establishing the first near-minimax optimal nonparametric regression risk bound for ReLU networks under practical optimization algorithms and designing a provably effective adaptive degree selection procedure. The resulting sample complexity scales as Θ(log(4/δ)·d^{k₀}/ε), and the regression risk achieves log(4/δ)·Θ(d^{k₀}/n), significantly improving upon existing approaches.

This study investigates the theoretical foundations of operator learning, with a focus on convergence rates and fundamental statistical limits. By integrating tools from statistical learning theory, approximation theory, and the framework of holomorphic operators, the work establishes a unified error analysis framework to systematically derive generalization error bounds for empirical risk minimization. Under generalized regularity conditions, it further establishes minimax-optimal statistical lower bounds, revealing an intrinsic trade-off between sample complexity and model approximation capacity. The analysis delineates current theoretical boundaries in operator learning and identifies several key open problems, offering new perspectives to guide future theoretical advances in the field.

This paper addresses the fundamental question of why overparameterized neural networks—despite achieving zero training error on sufficiently many samples—still exhibit strong generalization performance, particularly when interpolation solutions are selected randomly. Method: We introduce tools from algebraic geometry to characterize the geometric structure of the solution set (an algebraic variety) in parameter space, analyzing its dimension and irreducible components. Contribution/Results: We rigorously prove that, once the number of training samples exceeds a threshold determined by the intrinsic dimension of the parameter space and the algebraic structure of the model class, the generalization error of a uniformly random interpolator vanishes almost surely. Crucially, this result does not rely on assumptions about optimization dynamics (e.g., SGD’s implicit bias), offering a geometric-probabilistic explanation for the high generalization capability of large models. It establishes that generalization arises fundamentally from the structural properties of the interpolation solution set—not from algorithm-specific inductive biases.

This study investigates the conjecture that minimal-width polynomial neural networks (PNNs) must exhibit a unimodal width profile across hidden layers. By integrating advanced search strategies, recursive dimensional bound analysis, and symbolic computation, the authors construct the first counterexample that definitively refutes this unimodality assumption. The discovered architecture features a notably large defective substructure, challenging the prevailing belief that optimal PNNs are dominated by small defects. This finding reveals richer and more complex patterns in the optimal distribution of hidden layer widths, thereby expanding our understanding of the representational capacity of polynomial neural networks.