Experimental Evaluation of Dynamic Topic Modeling Algorithms

2025-08-01

AI Summary

Existing dynamic topic modeling lacks a unified, quantitative evaluation framework—particularly for characterizing the regularity and interpretability of temporal topic evolution. Method: We systematically evaluate mainstream dynamic topic models (e.g., DTM, DCTM, STTM) on large-scale social media time-series text corpora, proposing a novel metric—Evolutionary Consistency Score (ECS)—that jointly integrates time-series pattern recognition with topic coherence constraints. Contribution/Results: ECS is the first metric to jointly quantify topic stability, evolutionary sensitivity, and semantic interpretability. Experiments demonstrate that ECS effectively discriminates model performance: it reveals DTM’s bias in long-term evolutionary modeling and STTM’s superiority in detecting bursty topics. This work fills a critical gap in standardized evaluation for dynamic topic models and provides empirical grounding for algorithm selection and improvement.

Technology Category

• Machine Learning: Evaluation and Analysis
• Natural Language Processing: Interpretability, Analysis, and Evaluation of NLP Models
• Data Mining & Knowledge Management: Mining of Spatial, Temporal or Spatio-Temporal Data

Application Category

• Web Mining and Content Analysis: Models for Web evolution
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• User Modeling, Personalization and Recommendation: Metrics for user behavior and evaluating success

Abstract

The amount of text generated daily on social media is gigantic and analyzing this text is useful for many purposes. To understand what lies beneath a huge amount of text, we need dependable and effective computing techniques from self-powered topic models. Nevertheless, there are currently relatively few thorough quantitative comparisons between these models. In this study, we compare these models and propose an assessment metric that documents how the topics change in time.

Problem

• Evaluating dynamic topic modeling algorithms for social media text
• Comparing performance of self-powered topic models quantitatively
• Proposing metrics to track topic evolution over time

Innovation

• Dynamic topic modeling algorithms comparison
• Proposed assessment metric for topic changes
• Analyzed social media text at scale