Technology Category
Data Mining & Knowledge Management: Semantic WebNatural Language Processing: Discourse, Pragmatics & Argument MiningMachine Learning: Large Multimodal Models (LMMs)
Multivariate Gaussian Topic Modelling: A novel approach to discover topics with greater semantic coherence
🤖 AI Summary
Traditional topic models such as LDA suffer from poor topic interpretability and low semantic coherence when applied to short texts. To address this, we propose the Multivariate Gaussian Distribution (MGD) topic model, which represents topics as multivariate Gaussian distributions and documents as Gaussian mixture models, with parameters learned via the EM algorithm. Crucially, MGD innovatively identifies highly discriminative keywords through joint analysis of mean vectors and covariance structures, enabling interpretable topic labeling. Experiments on synthetic data and real-world cybersecurity incident reports demonstrate that MGD significantly improves topic semantic coherence—achieving an average Normalized Pointwise Mutual Information (NPMI) of 0.436, a 48.3% improvement over LDA’s 0.294. Moreover, MGD successfully uncovers critical latent themes—such as implicit safety risks in petrochemical plants—validating its effectiveness and practical utility in domain-specific short-text scenarios.
Technology Category
Application Category
📝 Abstract
An important aspect of text mining involves information retrieval in form of discovery of semantic themes (topics) from documents using topic modelling. While generative topic models like Latent Dirichlet Allocation (LDA) elegantly model topics as probability distributions and are useful in identifying latent topics from large document corpora with minimal supervision, they suffer from difficulty in topic interpretability and reduced performance in shorter texts. Here we propose a novel Multivariate Gaussian Topic modelling (MGD) approach. In this approach topics are presented as Multivariate Gaussian Distributions and documents as Gaussian Mixture Models. Using EM algorithm, the various constituent Multivariate Gaussian Distributions and their corresponding parameters are identified. Analysis of the parameters helps identify the keywords having the highest variance and mean contributions to the topic, and from these key-words topic annotations are carried out. This approach is first applied on a synthetic dataset to demonstrate the interpretability benefits vis-`a-vis LDA. A real-world application of this topic model is demonstrated in analysis of risks and hazards at a petrochemical plant by applying the model on safety incident reports to identify the major latent hazards plaguing the plant. This model achieves a higher mean topic coherence of 0.436 vis-`a-vis 0.294 for LDA.
Problem
Research questions and friction points this paper is trying to address.
Improves topic interpretability in text mining
Enhances performance in analyzing shorter texts
Identifies major latent hazards in safety reports
Innovation
Methods, ideas, or system contributions that make the work stand out.
Multivariate Gaussian Distributions model topics
EM algorithm identifies Gaussian parameters
Higher topic coherence than LDA achieved
🔎 Similar Papers
2024-04-02North American Chapter of the Association for Computational LinguisticsCitations: 2
