VisionClaw: Always-On AI Agents through Smart Glasses

2026-04-03

AI Summary

This work addresses the high interaction latency and operational complexity of conventional wearable devices, which stem from their inability to sustain continuous perception and immediate action. To overcome these limitations, the authors propose an always-on AI agent system deployed on Meta Ray-Ban smart glasses that, for the first time, enables persistent coupling between first-person visual perception and voice-driven task execution, allowing users to seamlessly delegate tasks in real-world settings. Built upon on-device sensing, the OpenClaw agent framework, voice triggering, and multimodal scheduling, the system establishes an in-situ, hands-free, context-driven interaction paradigm. Experimental results demonstrate that, compared to non-always-on or non-agent baselines, the proposed system significantly improves task completion efficiency, reduces interaction overhead, and encourages users to opportunistically initiate tasks more naturally during everyday activities.

Abstract

We present VisionClaw, an always-on wearable AI agent that integrates live egocentric perception with agentic task execution. Running on Meta Ray-Ban smart glasses, VisionClaw continuously perceives real-world context and enables in-situ, speech-driven action initiation and delegation via OpenClaw AI agents. Therefore, users can directly execute tasks through the smart glasses, such as adding real-world objects to an Amazon cart, generating notes from physical documents, receiving meeting briefings on the go, creating events from posters, or controlling IoT devices. We evaluate VisionClaw through a controlled laboratory study (N=12) and a longitudinal deployment study (N=5). Results show that integrating perception and execution enables faster task completion and reduces interaction overhead compared to non-always-on and non-agent baselines. Beyond performance gains, deployment findings reveal a shift in interaction: tasks are initiated opportunistically during ongoing activities, and execution is increasingly delegated rather than manually controlled. These results suggest a new paradigm for wearable AI agents, where perception and action are continuously coupled to support situated, hands-free interaction.

Problem

• always-on AI agents
• egocentric perception
• wearable AI
• situated interaction
• task delegation

Innovation

• always-on AI agent
• egocentric perception
• speech-driven interaction
• wearable intelligence
• perception-action coupling