Seeded Poisson Factorization: Leveraging domain knowledge to fit topic models

2025-03-04

AI Summary

Traditional unsupervised topic models struggle to align discovered topics with predefined domain concepts, resulting in limited interpretability and structural coherence. To address this, we propose a seed-word-guided Poisson factorization framework that, for the first time, encodes domain knowledge as a seed-driven prior over topic intensities and employs an adaptive mechanism to balance prior guidance with data-driven learning—ensuring robustness under imperfect seed specifications. The method leverages variational inference coupled with stochastic gradient optimization to achieve both computational efficiency and estimation accuracy. Experiments on Amazon user feedback data demonstrate that our model significantly outperforms existing guided topic models in topic interpretability, downstream text classification performance, and runtime efficiency. These results validate both its theoretical novelty—particularly in principled integration of weak supervision—and its practical utility for real-world applications requiring semantically grounded topic discovery.

Technology Category

• Machine Learning: Unsupervised & Self-Supervised Learning
• Reasoning under Uncertainty: Relational Probabilistic Models
• Natural Language Processing: Sentence-level Semantics, Textual Inference, etc.

Application Category

• Web Mining and Content Analysis: Topic discovery and tracking
• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Topic models are widely used for discovering latent thematic structures in large text corpora, yet traditional unsupervised methods often struggle to align with predefined conceptual domains. This paper introduces Seeded Poisson Factorization (SPF), a novel approach that extends the Poisson Factorization framework by incorporating domain knowledge through seed words. SPF enables a more interpretable and structured topic discovery by modifying the prior distribution of topic-specific term intensities, assigning higher initial rates to predefined seed words. The model is estimated using variational inference with stochastic gradient optimization, ensuring scalability to large datasets. We apply SPF to an Amazon customer feedback dataset, leveraging predefined product categories as guiding structures. Our evaluation demonstrates that SPF achieves superior classification performance compared to alternative guided topic models, particularly in terms of computational efficiency and predictive performance. Furthermore, robustness checks highlight SPF's ability to adaptively balance domain knowledge and data-driven topic discovery, even in cases of imperfect seed word selection. These results establish SPF as a powerful and scalable alternative for integrating expert knowledge into topic modeling, enhancing both interpretability and efficiency in real-world applications.

Problem

• Improves topic model alignment with predefined conceptual domains.
• Enhances interpretability and structure in topic discovery using seed words.
• Ensures scalability and efficiency in large dataset applications.

Innovation

• Incorporates domain knowledge via seed words
• Uses variational inference with stochastic optimization
• Enhances topic model interpretability and efficiency