Covariate-dependent hierarchical Dirichlet processes

2024-07-02

AI Summary

This paper addresses density estimation and cross-group clustering for multiple related datasets. We propose a covariate-informed hierarchical Bayesian nonparametric model. Methodologically, we innovatively couple the Hierarchical Dirichlet Process (HDP) with the Dependent Dirichlet Process (DDP) to enable covariate-driven dynamic clustering structure modeling—the first such integration. We encode mixed-type covariates via kernel functions and resolve the intractable DDP normalization weight problem using a data augmentation strategy, enabling posterior inference via MCMC. The model supports flexible base distributions, substantially enhancing cross-group information sharing and quantification of population-level heterogeneity. Empirically, on single-cell RNA-seq data, it discovers novel, dynamically evolving cell subpopulations; on calcium imaging data, it identifies behaviorally interpretable temporal clusters of neural activity. These results demonstrate the model’s interpretability, practical utility, and superior performance in capturing structured heterogeneity across groups.

Technology Category

• Reasoning under Uncertainty: Relational Probabilistic Models
• Machine Learning: Bayesian Learning
• Cognitive Modeling & Cognitive Systems: Neural Spike Coding

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for heterogeneous, signed, attributed, multi-relational, temporal, higher-order, and annotated Web-related graphs
• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Semantics and Knowledge: Methods, algorithms and applications for the development of semantic models, knowledge graphs and other forms of structured data models with machine-interpretable semantics

Abstract

Bayesian hierarchical modelling is a natural framework to effectively integrate data and borrow information across groups. In this paper, we address problems related to density estimation and identifying clusters across related groups, by proposing a hierarchical Bayesian approach that incorporates additional covariate information. To achieve flexibility, our approach builds on ideas from Bayesian nonparametrics, combining the hierarchical Dirichlet process with dependent Dirichlet processes. The proposed model is widely applicable, accommodating multiple and mixed covariate types through appropriate kernel functions as well as different output types through suitable likelihoods. This extends our ability to discern the relationship between covariates and clusters, while also effectively borrowing information and quantifying differences across groups. By employing a data augmentation trick, we are able to tackle the intractable normalized weights and construct a Markov chain Monte Carlo algorithm for posterior inference. The proposed method is illustrated on simulated data and two real data sets on single-cell RNA sequencing (scRNA-seq) and calcium imaging. For scRNA-seq data, we show that the incorporation of cell dynamics facilitates the discovery of additional cell subgroups. On calcium imaging data, our method identifies interpretable clusters of time frames with similar neural activity, aligning with the observed behavior of the animal.

Problem

• Estimating densities and clusters across related groups
• Incorporating covariate information in hierarchical Bayesian modeling
• Extending cluster-covariate relationship analysis with flexible nonparametrics

Innovation

• Hierarchical Bayesian modeling with covariate information
• Combines hierarchical and dependent Dirichlet processes
• MCMC algorithm for posterior inference via data augmentation