Structured Sparse Memory for Recurrent Reasoning

2026-09-27

AI Summary

This study addresses the limitations of recurrent models on ARC reasoning tasks, specifically their neglect of memory capacity and synthetic data augmentation. To overcome these challenges, we propose CHARM, a hybrid model that integrates recurrent reasoning, structured task memory, and test-time aggregation strategies. Furthermore, we introduce Combinatorial Sparse Embeddings (CoSE), an innovative design that reduces parameter count by 90% while enhancing performance, effectively balancing recurrence depth with learning horizon. Experimental results demonstrate that CHARM achieves pass@2 accuracies of 84.0% and 46.7% on the ARC-AGI-1 and ARC-AGI-2 benchmarks, respectively, exhibiting superior abstract generalization capabilities.

Abstract

Recurrent models trained from scratch have recently become competitive on ARC-style reasoning tasks, but the usual framing around small recurrent backbones overlooks two important parts of the system: task-conditioned memory and synthetic augmentation data. We study this regime through CHARM, a compact hybrid ARC model that combines recurrent reasoning with structured task memory, synthetic data, and inference-time aggregation. In existing approaches, task-conditioned memory supplies a large hidden source of capacity, reaching more than 30x the size of the recurrent backbone. We introduce a compositional sparse embedding (CoSE) for task conditioning that reduces learned task-memory parameters by over 90% while improving pass@2 in controlled ARC ablations. For the recurrent backbone, recurrent depth helps only when balanced with learning horizon. Combining these ingredients, our system reaches 84% pass@2 on ARC-AGI-1 and 46.7% pass@2 on ARC-AGI-2 public evaluation. The benefits of structured memory also generalize to unseen puzzles and other domains. Our code, dataset, and model checkpoints are available at https://github.com/water-vapor/charm.

Problem

• recurrent reasoning
• task-conditioned memory
• ARC-style reasoning
• synthetic augmentation data

Innovation

• Compositional Sparse Embedding
• Recurrent Reasoning
• Structured Task Memory
• Synthetic Data Augmentation
• ARC-AGI