Problem
Research questions and friction points this paper is trying to address.
smart glasses
event-based vision
on-device ML
power constraints
embedded AI
OpenGlass: Open-Source Smart Glasses for On-Device Event-Based Gesture Recognition
🤖 AI Summary
This work addresses the limitations of current smart glasses, which are constrained by power consumption, memory, and computational capacity in compact form factors, and lack an open-source platform supporting event-based vision and embedded machine learning. The authors propose the first open-source, modular smart glasses architecture, featuring a flexible FPC interposer and an event-driven wake-up mechanism that enables plug-and-play sensor integration without requiring PCB redesign. The system integrates a Prophesee GENX320 event camera, a GAP9 RISC-V SoC, an nRF5340 co-processor, and a configurable PMIC. Leveraging an R(2+1)D network with polarity-separated event histogram features, the prototype achieves 11.8 hours of battery life on a 200 mAh cell, 83.94% cross-user gesture recognition accuracy, and a 33.9 ms end-to-end latency.
📝 Abstract
Smart eyewear enables unobtrusive, context-aware interaction through multimodal sensors and on-device intelligence, but is severely limited by power, memory, and compute constraints in a compact form factor. Open-hardware platforms supporting event-based vision and embedded ML at this scale are rare. This work introduces an open-source smart glasses platform for rapid prototyping of novel sensors and algorithms. Its modular design uses a flexible FPC interposer to support both event-based and frame-based cameras without full PCB redesign. A hardware-software co-designed power management system combines a configurable PMIC with event-driven wake-up via an nRF5340 coordinator, keeping the GAP9 RISC-V SoC powered down between inferences. The prototype achieves up to 11.8 hours of continuous on-device ML from a 200 mAh battery. As a demonstration, an egocentric hand gesture recognition pipeline was evaluated on the LynX dataset using polarity-separated event histograms from a Prophesee GENX320 camera. R(2+1)D achieved the best cross-subject accuracy of 83.94\% (macro F1 = 0.781) under leave-two-subjects-out validation, with 33.9 ms end-to-end latency on the GAP9. Temporal augmentation and removal of ambiguous classes provided the largest gains (+8.9 pp). All hardware designs, firmware, and models are released open source.
Innovation
Methods, ideas, or system contributions that make the work stand out.
event-based vision
on-device ML
modular smart glasses
hardware-software co-design
low-power embedded system
🔎 Similar Papers
No similar papers found.
💼 Related Jobs
