MemoAct: Atkinson-Shiffrin-Inspired Memory-Augmented Visuomotor Policy for Robotic Manipulation

2026-03-19

AI Summary

This work addresses the challenge that existing memory-augmented methods struggle to simultaneously achieve precise task-state tracking and robust long-term memory retention. Inspired by the Atkinson-Shiffrin memory model, we propose a hierarchical memory architecture for robotic policies that integrates human-like multi-level memory mechanisms. The architecture employs a lossless short-term memory buffer to ensure operational precision while leveraging a compressed long-term memory representation to maintain temporal consistency. Evaluated on our newly introduced MemoryRTBench benchmark, the proposed approach significantly outperforms Markovian baselines and state-of-the-art history-aware strategies in both simulated and real-world environments, demonstrating superior performance in task-state tracking and long-horizon memory-intensive tasks.

Technology Category

• Intelligent Robots: Learning & Optimization for ROB
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Cognitive Modeling & Cognitive Systems: Agent Architectures

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Memory-augmented robotic policies are essential in handling memory-dependent tasks. However, existing approaches typically rely on simple observation window extensions, struggling to simultaneously achieve precise task state tracking and robust long-horizon retention. To overcome these challenges, inspired by the Atkinson-Shiffrin memory model, we propose MemoAct, a hierarchical memory-based policy that leverages distinct memory tiers to tackle specific bottlenecks. Specifically, lossless short-term memory ensures precise task state tracking, while compressed long-term memory enables robust long-horizon retention. To enrich the evaluation landscape, we construct MemoryRTBench based on RoboTwin 2.0, specifically tailored to assess policy capabilities in task state tracking and long-horizon retention. Extensive experiments across simulated and real-world scenarios demonstrate that MemoAct achieves superior performance compared to both existing Markovian baselines and history-aware policies. The project page is \href{https://tlf-tlf.github.io/MemoActPage/}{available}.

Problem

• memory-augmented policy
• task state tracking
• long-horizon retention
• robotic manipulation
• visuomotor control

Innovation

• memory-augmented policy
• hierarchical memory
• Atkinson-Shiffrin model
• long-horizon retention
• task state tracking