Learnable & Interpretable Model Combination in Dynamic Systems Modeling

2024-06-12

arXiv.org

AI Summary

Integrating physics-based and machine learning models for dynamic system modeling remains challenging due to difficulties in unifying multi-model representations, handling algebraic loops, and managing discontinuous events within a coherent framework. Method: This paper proposes a learnable and interpretable hybrid modeling paradigm built upon a novel wildcard architecture—the first to enable unified symbolic representation of algebraic, discrete, and differential equations—supporting end-to-end differentiable joint optimization of physics-informed and data-driven components. Grounded in systems theory and symbolic modeling principles, the approach inherently avoids algebraic loops and explicitly models discontinuities. Contribution/Results: Experiments demonstrate that the framework automatically identifies and resolves diverse dual-model compositions, achieving significant improvements over state-of-the-art methods in prediction accuracy, model interpretability, and cross-scenario generalizability.

Technology Category

• Constraint Satisfaction and Optimization: Mixed Discrete/Continuous Optimization
• Machine Learning: Neuro-Symbolic Learning
• Search and Optimization: Mixed Discrete/Continuous Search

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

During modeling of dynamical systems, often two or more model architectures are combined to obtain a more powerful or efficient model regarding a specific application area. This covers the combination of multiple machine learning architectures, as well as hybrid models, i.e., the combination of physical simulation models and machine learning. In this work, we briefly discuss which types of model are usually combined in dynamical systems modeling and propose a class of models that is capable of expressing mixed algebraic, discrete, and differential equation-based models. Further, we examine different established, as well as new ways of combining these models from the point of view of system theory and highlight two challenges - algebraic loops and local event functions in discontinuous models - that require a special approach. Finally, we propose a new wildcard architecture that is capable of describing arbitrary combinations of models in an easy-to-interpret fashion that can be learned as part of a gradient-based optimization procedure. In a final experiment, different combination architectures between two models are learned, interpreted, and compared using the methodology and software implementation provided.

Problem

• Dynamic System Modeling
• Algebraic Loops
• Model Discontinuities

Innovation

• Algebraic-Differential-Discrete Equations
• Flexible Model Integration
• Optimization Learning Architecture