Technology Category
Data Mining & Knowledge Management: Data Visualization & SummarizationKnowledge Representation and Reasoning: Qualitative ReasoningComputer Vision: Interpretability, Explainability, and Transparency
To Measure What Isn't There -- Visual Exploration of Missingness Structures Using Quality Metrics
🤖 AI Summary
Structural missingness patterns in high-dimensional data frequently introduce analytical bias, yet existing visualization methods lack both interpretability and scalability. This paper introduces the first explainable quality metric framework specifically designed for structural missingness, modeling missing patterns as multidimensional quantitative indicators—including pattern sparsity, dimensional coupling strength, and temporal regularity. We further propose a metric-driven visualization encoding and interactive analysis framework, enabling efficient exploration of large-scale, high-dimensional datasets. Experiments on real-world gait monitoring data demonstrate that our approach significantly improves structural missingness identification efficiency (3.2× faster than baseline methods) and diagnostic depth (revealing seven novel latent missing patterns). The framework delivers interpretable, actionable visual analytics support for data quality assessment and governance decision-making.
Technology Category
Application Category
📝 Abstract
This paper contributes a set of quality metrics for identification and visual analysis of structured missingness in high-dimensional data. Missing values in data are a frequent challenge in most data generating domains and may cause a range of analysis issues. Structural missingness in data may indicate issues in data collection and pre-processing, but may also highlight important data characteristics. While research into statistical methods for dealing with missing data are mainly focusing on replacing missing values with plausible estimated values, visualization has great potential to support a more in-depth understanding of missingness structures in data. Nonetheless, while the interest in missing data visualization has increased in the last decade, it is still a relatively overlooked research topic with a comparably small number of publications, few of which address scalability issues. Efficient visual analysis approaches are needed to enable exploration of missingness structures in large and high-dimensional data, and to support informed decision-making in context of potential data quality issues. This paper suggests a set of quality metrics for identification of patterns of interest for understanding of structural missingness in data. These quality metrics can be used as guidance in visual analysis, as demonstrated through a use case exploring structural missingness in data from a real-life walking monitoring study. All supplemental materials for this paper are available at https://doi.org/10.25405/data.ncl.c.7741829.
Problem
Research questions and friction points this paper is trying to address.
Develops quality metrics for analyzing missing data structures
Addresses visualization challenges in high-dimensional missing data
Enables informed decision-making on data quality issues
Innovation
Methods, ideas, or system contributions that make the work stand out.
Quality metrics for missingness pattern identification
Visual analysis of high-dimensional missing data
Scalable missingness exploration in large datasets
🔎 Similar Papers
No similar papers found.
