Discrete-time Markov chains with random observation times

2025-05-19

AI Summary

This paper addresses the estimation of transition matrices for discrete-time Markov chains under sparse, randomly timed observations: each sample path contains only two observations, and transition probabilities are covariate-dependent and time-inhomogeneous. To tackle this highly challenging ultra-sparse setting, we propose, for the first time, an online nonparametric estimator based on kernel smoothing—integrating stochastic process modeling with empirical process theory to establish a unified framework for convergence rate analysis. We prove that the estimator achieves uniform consistency under mild regularity conditions. Simulation studies demonstrate its high accuracy and robustness even under low-frequency observation schemes. The key innovation lies in circumventing the conventional assumption of multi-step observations, enabling provably optimal estimation from merely two random observations per trajectory, while accommodating covariate-driven, time-varying transition dynamics.

Technology Category

• Reasoning under Uncertainty: Stochastic Optimization
• Machine Learning: Kernel Methods
• Intelligent Robots: State Estimation

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Economics, Online Markets and Human Computation: Sustainability of Web economics
• Web Mining and Content Analysis: Models for Web evolution

Abstract

We propose a new approach for estimating the finite dimensional transition matrix of a Markov chain using a large number of independent sample paths observed at random times. The sample paths may be observed as few as two times, and the transitions are allowed to depend on covariates. Simple and easy to update kernel estimates are proposed, and their uniform convergence rates are derived. Simulation experiments show that our estimation approach performs well.

Problem

• Estimating Markov chain transition matrix with random observation times
• Handling sparse observations as few as two time points
• Incorporating covariate-dependent transitions in estimation

Innovation

• Estimates Markov chain transition matrix randomly
• Uses kernel methods for easy updates
• Derives uniform convergence rates effectively