Evaluating quality in synthetic data generation for large tabular health datasets

2026-04-17

AI Summary

This study addresses the lack of standardized evaluation protocols in existing methods for synthesizing health tabular data. To this end, it systematically assesses the performance of seven prominent generative models across four health datasets of varying scales, employing consistent hyperparameter tuning and joint distribution fidelity metrics to ensure a fair comparison. The work introduces a novel, unified evaluation framework that integrates multidimensional quantitative metrics with visual analytics, complemented by domain-informed medical interpretation. Through this approach, the study uncovers critical limitations of current models in adhering to clinical constraints and provides a reproducible, interpretable foundation for selecting appropriate synthetic data generators in healthcare applications.

Technology Category

• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Cognitive Modeling & Cognitive Systems: Computational Creativity
• Machine Learning: Evaluation and Analysis

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Social Networks and Social Media: Generative AI / large language models and their impact on social systems
• Web Mining and Content Analysis: Web data generation and simulation

Abstract

There is no consensus in the field of synthetic data on concise metrics for quality evaluations or benchmarks on large health datasets, such as historical epidemiological data. This study presents an evaluation of seven recent models from major machine learning families. The models were evaluated using four different datasets, each with a distinct scale. To ensure a fair comparison, we systematically tuned the hyperparameters of each model for each dataset. We propose a methodology for evaluating the fidelity of synthesized joint distributions, aligning metrics with visualization on a single plot. This method is applicable to any dataset and is complemented by a domain-specific analysis of the German Cancer Registries' epidemiological dataset. The analysis reveals the challenges models face in strictly adhering to the medical domain. We hope this approach will serve as a foundational framework for guiding the selection of synthesizers and remain accessible to all stakeholders involved in releasing synthetic datasets.

Problem

• synthetic data
• quality evaluation
• health datasets
• benchmarking
• tabular data

Innovation

• synthetic data evaluation
• tabular health data
• joint distribution fidelity
• hyperparameter tuning
• domain-specific benchmarking