M$^3$-Gen: Interpretable Multimodal Generation of Gene Expression Profiles Using Clinical and Imaging Data

2026-07-23

AI Summary

This study addresses the challenges posed by the high cost and privacy constraints associated with acquiring gene expression data, which hinder multimodal biomedical research. To overcome these limitations, the authors propose a generative adversarial network that integrates histopathology images with clinical metadata to synthesize biologically meaningful gene expression profiles. The framework leverages contrastive learning, multimodal embedding fusion, and attention mechanisms to construct a unified latent representation. Notably, it introduces an attention-based intrinsic interpretability module capable of identifying tissue regions that drive specific gene expression patterns. Experiments on The Cancer Genome Atlas (TCGA) dataset demonstrate that the generated expression data are not only realistic but also functionally coherent, thereby validating both the efficacy and interpretability of the proposed approach.

Abstract

Integrating heterogeneous biomedical data, including clinical metadata, histopathology images, and molecular profiles, is crucial for comprehensive disease understanding. However, gene expression data acquisition remains constrained by high costs and privacy concerns, limiting its use in multimodal research and AI-driven applications. We present MultiModal Molecular Generation (M$^3$-Gen), a novel framework for the generation of gene expression profiles by conditioning a Generative Adversarial Network on histopathology images and clinical metadata. M$^3$-Gen learns a unified latent representation from the clinical variables and the images, leveraging contrastive learning, and exploits the embeddings of the two modalities to guide a generative model in producing biologically coherent gene expression profiles. Evaluations on the TCGA dataset demonstrate that M$^3$-Gen generates realistic and functionally meaningful gene expression data. Importantly, by integrating multiple modalities in an attention-based mechanism, M$^3$-Gen provides intrinsic explainability: it allows the identification of which regions of the histopathology images most strongly influenced the generation of specific gene expression profiles, making the model's decisions interpretable by design.

Problem

• gene expression
• multimodal data
• data generation
• privacy constraints
• high cost

Innovation

• multimodal generation
• interpretable AI
• gene expression synthesis
• contrastive learning
• attention mechanism