Measuring Dataset Diversity from a Geometric Perspective

2026-02-10

AI Summary

Existing diversity metrics for datasets predominantly rely on statistical distributions or entropy, often overlooking the intrinsic geometric structure of the data. This work introduces persistence landscapes (PLs)—a tool from topological data analysis—into diversity assessment, offering a geometric perspective to quantify structural diversity and establishing a direct link between geometric features and diversity. The proposed PLDiv metric is grounded in rigorous theoretical foundations and exhibits strong interpretability. Empirical evaluations across multimodal settings demonstrate its robustness and reliability, positioning it as a novel paradigm for dataset construction, augmentation, and evaluation.

Technology Category

• Machine Learning: Learning with Manifolds
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning
• Data Mining & Knowledge Management: Mining of Visual, Multimedia & Multimodal Data

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Economics, Online Markets and Human Computation: Fairness, privacy, and diversity in economic environments
• User Modeling, Personalization and Recommendation: Metrics for user behavior and evaluating success

Abstract

Diversity can be broadly defined as the presence of meaningful variation across elements, which can be viewed from multiple perspectives, including statistical variation and geometric structural richness in the dataset. Existing diversity metrics, such as feature-space dispersion and metric-space magnitude, primarily capture distributional variation or entropy, while largely neglecting the geometric structure of datasets. To address this gap, we introduce a framework based on topological data analysis (TDA) and persistence landscapes (PLs) to extract and quantify geometric features from data. This approach provides a theoretically grounded means of measuring diversity beyond entropy, capturing the rich geometric and structural properties of datasets. Through extensive experiments across diverse modalities, we demonstrate that our proposed PLs-based diversity metric (PLDiv) is powerful, reliable, and interpretable, directly linking data diversity to its underlying geometry and offering a foundational tool for dataset construction, augmentation, and evaluation.

Problem

• dataset diversity
• geometric structure
• topological data analysis
• persistence landscapes
• diversity metrics

Innovation

• topological data analysis
• persistence landscapes
• dataset diversity
• geometric structure
• diversity metric