Personal AI Agent for Camera Roll VQA

2026-06-03

AI Summary

This work addresses the challenge of enabling AI assistants to efficiently comprehend and retrieve visual information from users’ massive personal photo collections to answer diverse queries. To this end, the authors propose a novel long-context reasoning paradigm tailored for personalized visual memory, which integrates hierarchical memory architecture with lightweight navigation tools to jointly capture fine-grained visual details and user-specific contextual consistency, thereby overcoming limitations of conventional text-centric long-context processing. Evaluated on CamRoll—a large-scale, human-annotated dataset comprising 31,476 images from 50 users and 2,500 question-answer pairs—the proposed CamRoll-Agent significantly outperforms existing baselines, demonstrating superior performance across tasks ranging from factual retrieval to open-ended recommendations.

Abstract

We study the personal camera roll visual question answering setting. In this setting, a conversational AI assistant can access a user's personal camera roll and retrieve relevant photos to answer queries, ranging from simple factual questions (e.g., ``Name of the food I tried yesterday?'') to more open-ended ones (e.g., ``Recommend some dishes I have never eaten before''). Given the vast nature of the personal camera roll (i.e., multiple years, hundreds to thousands of photos), a successful AI assistant needs to understand a long-horizon, highly personalized visual content stream in order to navigate and locate the correct and/or relevant information. To support this, we collect and manually annotate questions that mimic real-world usage. The final dataset, camroll, contains 50 users, 31,476 images, and 2,500 QA pairs. We further design camroll-agent, a conversational AI agent equipped with hierarchical memory and a minimal set of tools for efficient navigation over large, personalized visual memory. Experimental results show that camroll-agent outperforms numerous baselines and methods for long-context understanding AI agents system. Together, the camroll dataset and camroll-agent highlight the gap in AI agents' long-context reasoning: personalized visual memory requires different approaches from standard long-context textual memory, especially when consistency, visual details, and user-specific context are present.

Problem

• personal camera roll
• visual question answering
• long-context reasoning
• personalized visual memory
• AI agent

Innovation

• personalized visual memory
• camera roll VQA
• hierarchical memory
• long-context reasoning
• conversational AI agent