MemLife: Curating and Reasoning over Long-Term Egocentric Video Memories

2026-09-30

AI Summary

This study addresses the performance bottlenecks in long-term first-person video memory compression caused by evidence loss and retrieval competition. To overcome these challenges, we construct a multimodal memory system that generates entity-anchored text segments and design a temporally indexed proxy reader to enable efficient retrieval and reasoning. Furthermore, we introduce MemOpt, a novel reinforcement learning framework that optimizes memory writing strategies, significantly enhancing the faithfulness, informativeness, and retrievability of stored memories. Extensive evaluations across four long-horizon benchmarks demonstrate that our system achieves training-free performance improvements of 4.6%–12.0%, with MemOpt providing additional gains of 2.7%–5.0%.

Abstract

Long-term egocentric video enables personalized AI assistants to reason about daily life. However, as video histories grow to hundreds of hours spanning months or years, reprocessing raw clips for every query becomes computationally prohibitive. Memory systems offer a scalable alternative by compacting videos into text representations, but often fail on practical benchmarks: either the memory does not preserve key evidence, or the retriever fails to locate relevant entries due to retrieval competition in growing search spaces. To address these challenges, we introduce MemLife, a multimodal memory system that constructs entity-grounded, first-person text episodes and retrieves them via a time-indexed agentic reader. Without training or query-time video access, MemLife improves over the strongest training-free baseline by 4.6--12.0% across four long-horizon benchmarks. To further improve memory quality, we propose MemOpt, a reinforcement learning framework that optimizes the memory writer to produce faithful, informative, and retrievable memories. MemOpt consistently improves MemLife by 2.7--5.0% across different video and question distributions, with gains that generalize across writer and reader backbones and memory systems.

Problem

• Egocentric Video
• Long-term Memory
• Memory Retrieval
• Video Reasoning

Innovation

• Egocentric Video
• Multimodal Memory System
• Agentic Retrieval
• Reinforcement Learning
• Memory Optimization