Learning Mixtures of Linear Dynamical Systems (MoLDS) via Hybrid Tensor-EM Method

2025-10-07

AI Summary

To address poor identifiability and non-robust estimation of Mixture of Linear Dynamic Systems (MoLDS) under complex noise, this paper proposes the Hybrid Tensor-EM algorithm. It constructs moment tensors from input-output data to ensure global identifiability and embeds a Kalman filter to enable closed-form parameter updates for LDS components within the EM framework. The method achieves both theoretical identifiability guarantees and computational efficiency. On synthetic data, it significantly improves parameter recovery accuracy and robustness to noise. Applied to neural recordings from primate somatosensory cortex, it successfully uncovers distinct dynamical subsystems corresponding to different movement directions—without supervision—and yields results highly consistent with single-condition supervised models. This demonstrates its effectiveness and biological interpretability in real-world neural data analysis.

Technology Category

• Machine Learning: Mixture of Experts (MoE)
• Intelligent Robots: State Estimation
• Cognitive Modeling & Cognitive Systems: Neural Spike Coding

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

Mixtures of linear dynamical systems (MoLDS) provide a path to model time-series data that exhibit diverse temporal dynamics across trajectories. However, its application remains challenging in complex and noisy settings, limiting its effectiveness for neural data analysis. Tensor-based moment methods can provide global identifiability guarantees for MoLDS, but their performance degrades under noise and complexity. Commonly used expectation-maximization (EM) methods offer flexibility in fitting latent models but are highly sensitive to initialization and prone to poor local minima. Here, we propose a tensor-based method that provides identifiability guarantees for learning MoLDS, which is followed by EM updates to combine the strengths of both approaches. The novelty in our approach lies in the construction of moment tensors using the input-output data to recover globally consistent estimates of mixture weights and system parameters. These estimates can then be refined through a Kalman EM algorithm, with closed-form updates for all LDS parameters. We validate our framework on synthetic benchmarks and real-world datasets. On synthetic data, the proposed Tensor-EM method achieves more reliable recovery and improved robustness compared to either pure tensor or randomly initialized EM methods. We then analyze neural recordings from the primate somatosensory cortex while a non-human primate performs reaches in different directions. Our method successfully models and clusters different conditions as separate subsystems, consistent with supervised single-LDS fits for each condition. Finally, we apply this approach to another neural dataset where monkeys perform a sequential reaching task. These results demonstrate that MoLDS provides an effective framework for modeling complex neural data, and that Tensor-EM is a reliable approach to MoLDS learning for these applications.

Problem

• Modeling time-series data with diverse dynamics across trajectories
• Addressing noise and complexity challenges in neural data analysis
• Combining tensor identifiability guarantees with EM refinement

Innovation

• Hybrid Tensor-EM method combining tensor guarantees with EM refinement
• Moment tensor construction for global parameter recovery
• Kalman EM algorithm with closed-form parameter updates