Dynamic Sparsity: Challenging Common Sparsity Assumptions for Learning World Models in Robotic Reinforcement Learning Benchmarks

2025-11-11

AI Summary

This work investigates the validity of the dynamic sparsity assumption in world models for robotic reinforcement learning—specifically, whether (i) causal graphs are globally sparse, (ii) sparsity is state-dependent, and (iii) local dynamical changes are temporally sparse. Method: Leveraging high-fidelity dynamics data from MuJoCo Playground, we jointly apply causal graph discovery and local dynamical change detection to quantify sparsity across multidimensional state spaces and time. Contribution/Results: We find global sparsity is rare; instead, sparse connections concentrate heavily on contact-related degrees of freedom and transient events (e.g., collisions, detachment), exhibiting strong state dependence and temporal clustering. These findings challenge conventional globally sparse priors and advocate for *structural local sparsity*—a more physically grounded inductive bias wherein sparsity patterns follow identifiable spatiotemporal regularities. This insight informs principled world model design aligned with real-world physical dynamics.

Technology Category

• Reasoning under Uncertainty: Relational Probabilistic Models
• Knowledge Representation and Reasoning: Action, Change, and Causality
• Intelligent Robots: State Estimation

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Web Mining and Content Analysis: Models for Web evolution
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

The use of learned dynamics models, also known as world models, can improve the sample efficiency of reinforcement learning. Recent work suggests that the underlying causal graphs of such dynamics models are sparsely connected, with each of the future state variables depending only on a small subset of the current state variables, and that learning may therefore benefit from sparsity priors. Similarly, temporal sparsity, i.e. sparsely and abruptly changing local dynamics, has also been proposed as a useful inductive bias. In this work, we critically examine these assumptions by analyzing ground-truth dynamics from a set of robotic reinforcement learning environments in the MuJoCo Playground benchmark suite, aiming to determine whether the proposed notions of state and temporal sparsity actually tend to hold in typical reinforcement learning tasks. We study (i) whether the causal graphs of environment dynamics are sparse, (ii) whether such sparsity is state-dependent, and (iii) whether local system dynamics change sparsely. Our results indicate that global sparsity is rare, but instead the tasks show local, state-dependent sparsity in their dynamics and this sparsity exhibits distinct structures, appearing in temporally localized clusters (e.g., during contact events) and affecting specific subsets of state dimensions. These findings challenge common sparsity prior assumptions in dynamics learning, emphasizing the need for grounded inductive biases that reflect the state-dependent sparsity structure of real-world dynamics.

Problem

• Examining whether environment dynamics have sparse causal graphs in robotics
• Investigating if sparsity in dynamics is state-dependent rather than global
• Analyzing whether local system dynamics change sparsely over time

Innovation

• Challenging global sparsity assumptions in dynamics models
• Identifying state-dependent local sparsity in robot tasks
• Proposing grounded inductive biases for real dynamics