Are generative models fair? A study of racial bias in dermatological image generation

2025-01-20

AI Summary

This study investigates whether generative models—specifically variational autoencoders (VAEs)—exhibit race-associated skin-tone bias in dermatological image synthesis. Method: Leveraging the Fitzpatrick17k dataset, we train and evaluate a perception-loss-driven VAE, systematically quantifying reconstruction performance disparities across the six Fitzpatrick skin-type categories for the first time. Contribution/Results: We find significant performance degradation on darker skin tones (types IV–VI), with substantially higher reconstruction errors compared to lighter tones (types I–III). Crucially, standard uncertainty estimates—including reconstruction entropy and latent-variable variance—show no significant correlation with actual performance disparities, failing to expose fairness deficits. This work uncovers two critical issues in medical generative modeling: implicit representational bias and uncertainty–performance mismatch. We argue for fairness-aware uncertainty quantification frameworks and establish a methodological foundation for trustworthy AI in dermatology.

Technology Category

• Computer Vision: Bias, Fairness & Privacy
• Machine Learning: Deep Generative Models & Autoencoders
• Reasoning under Uncertainty: Uncertainty Representations

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI
• Social Networks and Social Media: Fairness and bias in social network and social media analysis

Abstract

Racial bias in medicine, particularly in dermatology, presents significant ethical and clinical challenges. It often results from the underrepresentation of darker skin tones in training datasets for machine learning models. While efforts to address bias in dermatology have focused on improving dataset diversity and mitigating disparities in discriminative models, the impact of racial bias on generative models remains underexplored. Generative models, such as Variational Autoencoders (VAEs), are increasingly used in healthcare applications, yet their fairness across diverse skin tones is currently not well understood. In this study, we evaluate the fairness of generative models in clinical dermatology with respect to racial bias. For this purpose, we first train a VAE with a perceptual loss to generate and reconstruct high-quality skin images across different skin tones. We utilize the Fitzpatrick17k dataset to examine how racial bias influences the representation and performance of these models. Our findings indicate that the VAE is influenced by the diversity of skin tones in the training dataset, with better performance observed for lighter skin tones. Additionally, the uncertainty estimates produced by the VAE are ineffective in assessing the model's fairness. These results highlight the need for improved uncertainty quantification mechanisms to detect and address racial bias in generative models for trustworthy healthcare technologies.

Problem

• Generative Image Models
• Racial Bias
• Uncertainty Estimation

Innovation

• Variational Autoencoders (VAEs)
• Perceptual Loss
• Bias Detection in Medical Imaging