A Simple Method for PMF Estimation on Large Supports

2025-10-16

AI Summary

This paper addresses the nonparametric estimation of multimodal, heavy-tailed probability mass functions (PMFs) supported on large discrete domains. We propose a graph signal processing–based smoothing framework: the empirical PMF is treated as a signal on a path graph; a data-adaptive, symmetric tridiagonal Laplacian operator is constructed; and denoising is performed by projecting the empirical PMF onto the low-dimensional subspace spanned by the eigenvectors associated with the smallest eigenvalues. Final estimates are obtained via truncation and renormalization. The method is fully automatic—requiring no manual hyperparameter tuning—and incorporates a theoretically grounded dimension-selection mechanism. Extensive experiments on synthetic and real heavy-tailed datasets demonstrate that our approach effectively suppresses sampling noise while preserving multimodal structure with high fidelity. It outperforms logspline and Gaussian kernel density estimation in accuracy and is computationally efficient with controllable memory footprint, making it suitable for large-scale automated analysis.

Technology Category

• Machine Learning: Graph-based Machine Learning
• Reasoning under Uncertainty: Graphical Models
• Data Mining & Knowledge Management: Mining of Visual, Multimedia & Multimodal Data

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Security and Privacy: Large-scale security measurements
• User Modeling, Personalization and Recommendation: Practical large-scale studies of user experience

Abstract

We study nonparametric estimation of a probability mass function (PMF) on a large discrete support, where the PMF is multi-modal and heavy-tailed. The core idea is to treat the empirical PMF as a signal on a line graph and apply a data-dependent low-pass filter. Concretely, we form a symmetric tri-diagonal operator, the path graph Laplacian perturbed with a diagonal matrix built from the empirical PMF, then compute the eigenvectors, corresponding to the smallest feq eigenvalues. Projecting the empirical PMF onto this low dimensional subspace produces a smooth, multi-modal estimate that preserves coarse structure while suppressing noise. A light post-processing step of clipping and re-normalizing yields a valid PMF. Because we compute the eigenpairs of a symmetric tridiagonal matrix, the computation is reliable and runs time and memory proportional to the support times the dimension of the desired low-dimensional supspace. We also provide a practical, data-driven rule for selecting the dimension based on an orthogonal-series risk estimate, so the method "just works" with minimal tuning. On synthetic and real heavy-tailed examples, the approach preserves coarse structure while suppressing sampling noise, compares favorably to logspline and Gaussian-KDE baselines in the intended regimes. However, it has known failure modes (e.g., abrupt discontinuities). The method is short to implement, robust across sample sizes, and suitable for automated pipelines and exploratory analysis at scale because of its reliability and speed.

Problem

• Estimating multi-modal heavy-tailed probability mass functions on large discrete supports
• Preserving coarse structure while suppressing sampling noise in PMF estimation
• Providing automated reliable computation for large-scale exploratory data analysis

Innovation

• Data-dependent low-pass filter smoothes empirical PMF
• Eigenvectors of perturbed Laplacian enable noise suppression
• Automated dimension selection ensures minimal tuning requirements