SJEPA: Learning Elegant Latent Dynamics with Hybrid Symbolic-Neural Predictors

2026-08-04

AI Summary

This work addresses the lack of interpretability and parsimony in conventional joint embedding predictive architectures, which typically rely on black-box neural networks to learn dynamics. The authors propose SJEPA, a symbolic-neural hybrid transition model that eschews reconstruction by integrating symbolic rules with a regularized neural correction term. By incorporating a “minimal sufficient dynamics” principle, the framework jointly optimizes representation constraints and operator compression, enabling a controllable trade-off between symbolic simplicity and representation quality while preventing representational collapse. Experiments on a pendulum task demonstrate that the learned symbolic dynamics are more compact, yielding lower long-horizon prediction errors and reduced divergence. Notably, even under misspecified symbolic grammars, the correction regularizer preserves expressible symbolic mechanisms and guides the neural component to learn residual dynamics effectively.

Abstract

Joint-embedding predictive architectures learn abstract states by predicting target embeddings from context embeddings, but their transition models are typically opaque neural maps. We introduce SJEPA, a reconstruction-free JEPA framework that learns predictive representations whose induced dynamics admit compact symbolic descriptions. Its hybrid transition combines a symbolic law with a regularised neural correction for dynamics outside the selected grammar. The central principle is to learn the simplest adequate dynamics: representation constraints preserve informative, non-collapsed predictive coordinates, while operator compression favours low-complexity symbolic-neural transitions that remain predictively adequate. We formalise this principle through induced-dynamics complexity, analyse predictive-coordinate non-identifiability, and show that unconstrained operator compression creates a direct shortcut to representation collapse. The framework supports both alternating representation-equation learning and symbolic dynamics fitted to fixed representations. In controlled pendulum experiments, joint learning discovers substantially simpler symbolic dynamics with lower long-horizon rollout error and divergence than post-hoc fitting, while an unconstrained one-step diagnostic realises the predicted collapse shortcut. Under grammar misspecification, correction regularisation preserves the representable symbolic mechanism and directs the neural component towards residual dynamics. The results expose a controllable trade-off among predictive fidelity, representation quality, symbolic parsimony, and symbolic-neural allocation.

Problem

• predictive representations
• symbolic dynamics
• representation collapse
• hybrid symbolic-neural models
• induced-dynamics complexity

Innovation

• symbolic-neural hybrid
• predictive representation
• representation collapse
• operator compression
• induced-dynamics complexity