Innovation
Methods, ideas, or system contributions that make the work stand out.
Global Workspace
State-Space Models
Jacobian Lenses
Recurrent States
Sign-Guarded Steering
Residual Streams Read, Recurrent States Remember: The Global Workspace in Mamba Models
🤖 AI Summary
This study investigates the applicability of Global Workspace Theory to Mamba state space models. Employing Jacobian lenses, joint readouts, and symbol-guided steering interventions, we systematically analyze the representational mechanisms underlying residual streams and recurrent states. Our findings reveal that recurrent states function as complementary carriers within the workspace, exhibiting information persistence. Symbol-guided steering confirms their capacity to independently support language access and content recovery. Furthermore, joint readouts substantially improve concept retrieval rates, with recurrent states outperforming residual streams across most tasks. By elucidating the causal role of recurrent states, this work offers novel perspectives for understanding the cognitive computational mechanisms inherent in state space models.
📝 Abstract
Can the global-workspace account of transformer representations extend to state-space language models? We fit Jacobian lenses to the residual streams and recurrent states of Mamba-1, Mamba-2 and Mamba-3, using the original 1000-prompt recipe. Joint residual--state readouts improve recovery of known intermediate concepts over the residual lens on at least five of six task families in every tested Mamba checkpoint. On Mamba-2, state alone exceeds residual and logit lenses on all six families; a normalised joint readout improves on both components on five. Temporal maps and word-list experiments show earlier content remaining state-readable as residual visibility changes. We also propose sign-guarded steering, which improves target top-five success over coordinate exchange on matched verbal-report trials in five models. Recurrent state alone supports this verbal access. These gains do not extend consistently to relational answers: guarded edits often output the edited concept itself, and Mamba-3's joint edits can disrupt successful state-only redirection. Recurrent state thus provides a complementary carrier of workspace content, whose recovery, persistence and causal uses require separate measurements.
🔎 Similar Papers
No similar papers found.
