Do Agents Dream of False Memories? Black-box Visual Attacks on Long-term Memory in Multimodal AI Agents

2026-07-17

AI Summary

This work addresses a critical vulnerability in multimodal AI agents whose long-term memory, heavily reliant on visual inputs, is susceptible to black-box adversarial attacks that can implant false memories or alter existing ones. The authors propose Lucid, a novel framework that, under a strict black-box setting with access only to image inputs, generates imperceptible perturbations without requiring internal model details, retrieval encoders, or textual channels. Lucid enables two distinct attack types: context-aware memory poisoning and context-agnostic memory injection. Evaluated across five black-box systems—including graph-based architectures, LLM-based summarizers, and commercial multimodal memory systems—Lucid achieves attack success rates of 61.6% for memory poisoning and 58.4% for memory injection, thereby exposing fundamental structural weaknesses in current multimodal memory designs.

Abstract

Multimodal AI agents increasingly rely on persistent long-term memory to ground generation in past visual and textual episodes. We show that unconditional trust in visual data creates a critical vulnerability. We propose Lucid, a black-box adversarial framework that compromises multimodal memory pipelines under a strictly image-bounded threat model, requiring no access to the target MLLM, target retrieval encoder, or the text channel. Lucid crafts imperceptible perturbations to enable two distinct failure modes based on the availability of historical context: (1) Memory poisoning, an in-context attack where the adversarial image replaces a benign one whose content is reinforced by prior textual context, reliably corrupting visual recall and steering the agent toward attacker-chosen narratives; (2) Memory injection, an out-of-context attack where the adversarial image replaces a benign one in a conversation turn devoid of prior textual grounding, causing the agent to generate attacker-influenced responses with no corrective signal from memory. We evaluate Lucid across various conversation domains and five black-box memory architectures, including graph-structured, LLM-summarized, and commercially deployed systems. Lucid achieves 61.6% ASR on poisoning and 58.4% ASR on injection, exposing a structural vulnerability in multimodal memory pipelines.

Problem

• multimodal AI agents
• long-term memory
• visual attacks
• false memories
• black-box adversarial

Innovation

• black-box attack
• multimodal memory
• adversarial perturbation
• memory poisoning
• memory injection