Efficient Bayesian Inference for Discretely Observed Continuous Time Markov Chains

2025-07-22

AI Summary

Bayesian inference for continuous-time Markov chains (CTMCs) from discrete-time observations faces two key challenges: intractable exact likelihoods and computational failure of existing methods in moderate-to-high-dimensional state spaces. To address these, we propose a scalable pseudo-likelihood framework that jointly models the transition probability matrix and the generator via a bi-orthogonal spectral decomposition—ensuring generator embeddability and enabling efficient Gibbs sampling. Theoretically, we establish posterior consistency for spectral parameters and verify the Bernstein–von Mises theorem. Computationally, the method achieves near-constant time complexity with respect to data size, dramatically improving inference efficiency and scalability in high-dimensional settings. Extensive simulations and real-data analyses demonstrate its robustness and flexibility across diverse scenarios.

Technology Category

• Machine Learning: Probabilistic Circuits and Graphical Models
• Reasoning under Uncertainty: Probabilistic Inference
• Planning, Routing, and Scheduling: Planning with Markov Models (MDPs, POMDPs)

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems
• Security and Privacy: Large-scale security measurements

Abstract

Inference for continuous-time Markov chains (CTMCs) becomes challenging when the process is only observed at discrete time points. The exact likelihood is intractable, and existing methods often struggle even in medium-dimensional state-spaces. We propose a scalable Bayesian framework for CTMC inference based on a pseudo-likelihood that bypasses the need for the full intractable likelihood. Our approach jointly estimates the probability transition matrix and a biorthogonal spectral decomposition of the generator, enabling an efficient Gibbs sampling procedure that obeys embeddability. Existing methods typically integrate out the unobserved transitions, which becomes computationally burdensome as the number of data or dimensions increase. The computational cost of our method is near-invariant in the number of data and scales well to medium-high dimensions. We justify our pseudo-likelihood approach by establishing theoretical guarantees, including a Bernstein-von Mises theorem for the probability transition matrix and posterior consistency for the spectral parameters of the generator. Through simulation and applications, we showcase the flexibility and robustness of our approach, offering a tractable and scalable approach to Bayesian inference for CTMCs.

Problem

• Inference for CTMCs with discrete observations is challenging
• Existing methods struggle in medium-dimensional state-spaces
• Full likelihood is intractable for CTMC inference

Innovation

• Bayesian framework with pseudo-likelihood for CTMCs
• Gibbs sampling for transition matrix and spectral decomposition
• Scalable method invariant to data size and dimensions