Anchored Variational Inference for Personalized Sequential Latent-State Models

2026-04-25

AI Summary

This work addresses the computational bottleneck arising from integrating out individual random effects in sequential latent variable models with heterogeneous subjects. To overcome this challenge, the authors propose the Anchored Variational Expectation-Maximization (AVEM) framework, which evaluates the posterior of local latent processes at representative anchor points of the individual random effects. This approach preserves local tractability while substantially reducing computational complexity. Theoretical analysis shows that the posterior mean constitutes a near-optimal anchor point and guarantees local monotonicity of the variational EM algorithm. A partially anchored variant is further introduced to accommodate varying degrees of posterior concentration. Experiments on mixture hidden Markov models and mixed-effects state-space models demonstrate that AVEM achieves superior parameter estimation accuracy alongside significant gains in computational efficiency.

Technology Category

• Machine Learning: Probabilistic Circuits and Graphical Models
• Reasoning under Uncertainty: Relational Probabilistic Models
• Search and Optimization: Mixed Discrete/Continuous Search

Application Category

• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Sequential latent-variable models with subject-specific random effects provide a flexible framework for modeling temporally structured data with both local latent dynamics and stable between-subject heterogeneity. In such models, conditional inference for the local latent process is often tractable, but integrating over subject-specific random effects can be computationally demanding. We propose an anchored variational inference framework for efficient approximate inference in this setting. The central idea is to replace the full conditional posterior of the local latent process with its evaluation at a representative value of the subject-specific latent effect, called the anchor point, thereby preserving tractable local inference while substantially reducing computational cost. This approximation is especially appealing in sequential settings, where the posterior distribution of the random effect becomes increasingly concentrated as the sequence length grows. Under suitable conditions, we show that the posterior mean is a nearly optimal anchor point and that the resulting anchored variational EM (AVEM) algorithm approximately preserves the local monotonicity behavior of standard variational inference. We instantiate the framework in two representative classes of sequential latent-variable models, namely mixed hidden Markov models and mixed-effects state-space models, derive the corresponding AVEM algorithms, and use simulation studies to indicate that the resulting methods achieve accurate estimation with substantial computational gains. We also discuss a partially anchored variant of the framework, in which only the components of the subject-specific latent effect whose posteriors are well concentrated are anchored.

Problem

• sequential latent-variable models
• subject-specific random effects
• variational inference
• computational efficiency
• posterior inference

Innovation

• anchored variational inference
• sequential latent-variable models
• subject-specific random effects
• variational EM
• computational efficiency