emb-diversity: A Tool for Embedding-Based Measurement of Data Diversity

2026-07-22

AI Summary

Existing approaches to evaluating data diversity are often confined to lexical-level metrics and lack standardization, hindering unified cross-dimensional and cross-data-type analysis. This work proposes a general-purpose embedding-based diversity measurement framework that is compatible with any embeddable data and arbitrary embedding models, enabling, for the first time, the unified quantification of diversity across multiple dimensions—including style, semantics, language, and speaker characteristics. By operating directly in embedding spaces, the framework addresses the longstanding gap in systematic diversity assessment and demonstrates strong generality and practical utility across diverse datasets.

Abstract

There is growing evidence that data diversity is crucial for developing fair and robust NLP models. However, current approaches to measure diversity remain inconsistent and fragmented: While there exist a number of tools for measuring the lexical diversity of texts, researchers lack standardized tools for quantifying diversity based on embeddings. Embedding-based diversity measures are highly flexible: They work with any embedding model and any data that can be embedded, and are thus applicable to many notions of diversity. With emb-diversity, we provide a comprehensive embedding-based diversity measurement tool, spanning a broad range of measures. We demonstrate its potential for several use cases: measuring the stylistic, semantic, language and speaker diversity of datasets. https://github.com/nlpsoc/emb-diversity/

Problem

• data diversity
• embedding-based measurement
• NLP fairness
• dataset diversity
• semantic diversity

Innovation

• embedding-based diversity
• data diversity
• NLP fairness
• diversity measurement
• emb-diversity