diversity metrics

This study addresses the critical need for quantifying dataset similarity in model generalization, transfer learning, simulation calibration, and two-sample testing. We systematically survey 118 similarity quantification methods and propose the first ten-dimensional classification framework, organizing approaches into seven technical categories: statistical distances (e.g., Wasserstein distance, Maximum Mean Discrepancy), kernel-based methods, information-theoretic measures, dimensionality-reduction embeddings, permutation tests, generative-model-based discriminators, and Gaussian process likelihood ratios. We develop a multi-dimensional evaluation system balancing theoretical guarantees, interpretability, and practical applicability, yielding a structured recommendation matrix aligned with task requirements and data characteristics. Furthermore, we introduce the first open-source, interactive tool for method selection—enabling real-time filtering and parameter configuration—to significantly enhance both selection efficiency and deployment suitability.

This study addresses the lack of a unified definition and quantification of training dataset diversity, which hinders accurate assessment of its impact on model robustness. Focusing on medical imaging, the work presents the first systematic comparison between reference-free and semantic diversity metrics. Leveraging the MorphoMNIST and PadChest datasets, the authors conduct a multidimensional analysis—integrating Fréchet Inception Distance (FID), AUC, semantic diversity measures, controlled perturbations, and clinical expert evaluations—to examine how diversity correlates with expert intuition, downstream performance, and training dynamics. The findings reveal that FID and semantic diversity more effectively predict model performance, whereas merely increasing the number of imaging device sources can inadvertently encourage models to rely on non-robust shortcut features, thereby exposing a critical pitfall in data diversity design.

Quantifying set diversity lacks a rigorous foundation, as mainstream metrics—e.g., distance-based, entropy-based, or coverage-based measures—fail to satisfy three desirable axioms: monotonicity, uniqueness, and continuity, undermining their reliability. Method: We establish the first axiomatized framework for diversity measurement, formally defining and proving the joint satisfiability of these axioms. Through systematic counterexample analysis, we demonstrate that existing approaches violate at least one axiom. We then construct an axiomatically complete metric and analyze its computational properties. Contribution/Results: We prove that any metric satisfying all three axioms is inherently NP-hard to compute, thereby establishing the “axiomatically complete yet computationally feasible” diversity measure as an open problem. This work provides the first rigorous benchmark for diversity evaluation and precisely characterizes the fundamental tension between theoretical soundness and practical computability in diversity quantification.

A lack of standardized, reproducible methods for quantifying textual diversity in large language models (LLMs) hinders rigorous evaluation of generation quality and cross-model or cross-corpus comparisons. Method: We propose the first systematic framework for text diversity evaluation, empirically validating convergent validity of diversity metrics and identifying a minimal, complete metric set—comprising compression ratio (zlib/lz4), long n-gram self-repetition rate, Self-BLEU, and BERTScore—that exhibits low inter-metric correlation and complementary multidimensional coverage. Contribution/Results: We release *diversity*, an open-source Python library enabling efficient computation and interactive visualization. Empirical analysis demonstrates that lightweight compression-based metrics robustly substitute for computationally expensive n-gram homogeneity scores. The framework substantially enhances interpretability, comparability, and practical utility of diversity assessment in LLM research.

To address the fairness–accuracy trade-off in AI-driven talent management—caused by implicit biases in training data—this paper introduces the first quality-diversity optimization framework for visual analytics based on MAP-Elites. The method explicitly constructs a fairness–accuracy Pareto front, enables interactive filtering of models satisfying minimum fairness thresholds, and establishes interpretable mappings between data bias and model behavior. Innovatively integrating CMA-MAP-Elites, multiple fairness metrics (statistical parity and equal opportunity), and bias heatmap visualization, the framework supports explainable, controllable fairness tuning. Evaluated on a real-world talent dataset, it achieves a 37% improvement in fairness while incurring less than a 2.1% drop in accuracy. Furthermore, it generates a two-dimensional behavioral atlas that visually exposes the fairness–accuracy trade-off boundary, facilitating transparent, human-in-the-loop decision-making.

This work addresses the challenges of rapidly growing nondominated solution sets and the tendency of search processes to become trapped in local regions of the Pareto front in multi-objective optimization. To overcome the limitations of existing approaches that focus solely on diversity in objective space, the authors propose a novel method emphasizing diversity in decision space. The approach incorporates a bounded archive mechanism based on Hamming distance and is rigorously evaluated against adaptive grid and hypervolume-based archiving strategies. Experimental results demonstrate that the proposed algorithm not only maintains a controllable archive size but also significantly enhances both the distribution and convergence efficiency of multi-objective local search, outperforming current state-of-the-art methods.

This study investigates whether prevailing diversity metrics genuinely capture model disagreement in large language model (LLM) ensembles or merely reflect individual model capabilities. Through controlled experiments on 31,900 subsets of 30 LLMs evaluated on MMLU-Pro and TruthfulQA, the authors systematically assess the predictive power of five diversity measures for majority-vote performance gains. Using Spearman correlation, linear regression, and determinant-based algebraic analysis, they find that most diversity metrics are highly collinear with model capability. After rigorously controlling for capability, only “strict diversity,” “disagreement,” and “double-failure” retain weak yet consistent and directionally aligned associations with ensemble failure rates. The results further reveal that simple majority voting surpasses the best individual model in only a small minority of subsets, highlighting the limited utility of current diversity metrics in LLM ensembling.

This work addresses the critical challenge in large-scale recommender systems of simultaneously maintaining recommendation quality, effectively controlling diversity, mitigating user churn, and managing computational complexity with million-scale item catalogs. The authors propose B-DivRec, a novel algorithm that uniquely integrates Determinantal Point Processes (DPPs) with a fuzzy de-redundancy mechanism to efficiently balance quality and diversity within recommendation batches. Furthermore, an adaptive online learning strategy is introduced to dynamically adjust the strength of diversity based on real-time user feedback. Extensive experiments on both synthetic and real-world datasets—spanning movie recommendation and drug repositioning tasks—demonstrate the method’s effectiveness: B-DivRec significantly enhances the synergy between recommendation diversity and positive user engagement while preserving scalability.

This study addresses the problem of maximizing committee diversity in multi-winner elections under constraints on either total approval scores or individual voter satisfaction. To this end, the authors propose two novel models based on approval voting and labeled candidates, each tailored to handle one type of constraint. They innovatively adapt and extend ecological diversity indices to the social choice setting, introducing a new index that satisfies desirable axiomatic properties and providing a complete axiomatic characterization. Through computational complexity analysis and empirical evaluation, the work delineates the tractability boundaries of the proposed models under mainstream voting rules and demonstrates that moderately relaxing the constraints can substantially enhance diversity outcomes.

Existing approaches to evaluating data diversity are often confined to lexical-level metrics and lack standardization, hindering unified cross-dimensional and cross-data-type analysis. This work proposes a general-purpose embedding-based diversity measurement framework that is compatible with any embeddable data and arbitrary embedding models, enabling, for the first time, the unified quantification of diversity across multiple dimensions—including style, semantics, language, and speaker characteristics. By operating directly in embedding spaces, the framework addresses the longstanding gap in systematic diversity assessment and demonstrates strong generality and practical utility across diverse datasets.

Existing diversity metrics for datasets predominantly rely on statistical distributions or entropy, often overlooking the intrinsic geometric structure of the data. This work introduces persistence landscapes (PLs)—a tool from topological data analysis—into diversity assessment, offering a geometric perspective to quantify structural diversity and establishing a direct link between geometric features and diversity. The proposed PLDiv metric is grounded in rigorous theoretical foundations and exhibits strong interpretability. Empirical evaluations across multimodal settings demonstrate its robustness and reliability, positioning it as a novel paradigm for dataset construction, augmentation, and evaluation.