ViSAGE: Constructing Self-Correcting Memories for Long-Form Video Understanding

2026-07-29

AI Summary

Existing multimodal agents often lose fine-grained entity identity cues in long-horizon video understanding due to memory compression and fragmented processing, and their reliance on vector similarity-based retrieval frequently leads to entity confusion and error propagation. To address these limitations, this work proposes an entity-centric self-correcting memory framework that preserves long-term identity information through cross-modal identity anchoring, refines historical records via a bidirectional memory refinement mechanism to retrospectively unify entity trajectories, and incorporates a multi-agent cross-verification strategy to enable reliable retrieval and rejection based on aligned identity–evidence pairs. Evaluated on long-form video understanding tasks, the proposed approach substantially outperforms the strongest baseline, achieving a 5.9% absolute improvement in accuracy.

Abstract

Multimodal agents operating in long-horizon environments must build and continually update multimedia memories to support entity-consistent, temporally grounded reasoning. However, existing agentic memory approaches often discard fine-grained dentity cues under aggressive compression and segment-wise processing. They also rely heavily on vector similarity retrieval, which can surface semantically related yet identity-mismatched evidence, leading to entity confusion, error propagation, and hallucinated answers. We propose ViSAGE, a multimodal agentic memory framework that constructs self-correcting, entity-centric memories. Specifically, ViSAGE anchors entity identity via cross-modal binding over long temporal ranges. It then applies bidirectional memory refinement to propagate delayed identity evidence, retroactively unifying historical records and improving future reasoning. We also introduce multi-agent cross-verification to assess retrieved evidence under an identity-evidence alignment onstraint, enabling abstention instead of unsupported answers when evidence is missing. Extensive results demonstrate that ViSAGE consistently outperforms the strongest baseline, achieving 5.9% higher accuracy.

Problem

• multimodal memory
• entity consistency
• long-form video understanding
• identity confusion
• temporal reasoning

Innovation

• self-correcting memory
• entity-centric reasoning
• cross-modal binding
• bidirectional memory refinement
• multi-agent cross-verification