State-Space Systems as Dynamic Generative Models

2024-04-12

arXiv.org

AI Summary

This work investigates the probabilistic dependence structure between input and output sequences in deterministic discrete-time state-space systems, focusing on the existence, uniqueness, and input-continuity of the induced output process. To relax conventional strong contraction assumptions, we introduce the *stochastic echo state property* (S-ESP), defined under a weak Wasserstein contraction condition. We rigorously establish that systems satisfying S-ESP admit a well-defined, causal, unique, and input-continuous probabilistic input–output mapping. Our analysis integrates tools from stochastic processes, dynamical systems stability theory, and Wasserstein metric geometry. This constitutes the first rigorous characterization of existence, uniqueness, and input continuity for a generalized echo state property. By transcending the limitations of purely deterministic modeling, our framework provides a broader theoretical foundation for state-space-based dynamic generative models, substantially extending both their applicability and theoretical depth.

Technology Category

• Machine Learning: Probabilistic Circuits and Graphical Models
• Reasoning under Uncertainty: Stochastic Optimization
• Cognitive Modeling & Cognitive Systems: Neural Spike Coding

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Web Mining and Content Analysis: Models for Web evolution
• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs

Abstract

A probabilistic framework to study the dependence structure induced by deterministic discrete-time state-space systems between input and output processes is introduced. General sufficient conditions are formulated under which output processes exist and are unique once an input process has been fixed, a property that in the deterministic state-space literature is known as the echo state property. When those conditions are satisfied, the given state-space system becomes a generative model for probabilistic dependences between two sequence spaces. Moreover, those conditions guarantee that the output depends continuously on the input when using the Wasserstein metric. The output processes whose existence is proved are shown to be causal in a specific sense and to generalize those studied in purely deterministic situations. The results in this paper constitute a significant stochastic generalization of sufficient conditions for the deterministic echo state property to hold, in the sense that the stochastic echo state property can be satisfied under contractivity conditions that are strictly weaker than those in deterministic situations. This means that state-space systems can induce a purely probabilistic dependence structure between input and output sequence spaces even when there is no functional relation between those two spaces.

Problem

• Study dependence structure in state-space systems
• Establish conditions for output process existence and uniqueness
• Generalize deterministic echo state property to stochastic cases

Innovation

• Probabilistic framework for state-space systems
• Echo state property under weaker conditions
• Continuous dependence using Wasserstein metric