Problem
Research questions and friction points this paper is trying to address.
on-device search
multi-modal search
Apple Silicon
local indexing
privacy-preserving retrieval
omni-macos: On-Device Omni-Modal Search on Apple Silicon
🤖 AI Summary
This work addresses the challenge of unifying text, code, documents, images, audio, and video in on-device multimodal search under stringent local compute and memory constraints. It presents the first end-to-end, full-modality multimodal search system deployed entirely on Apple Silicon Macs, integrating encoders, indexing, and storage while ensuring complete data locality. The system innovatively combines multimodal embeddings, quantized indexing, incremental re-encoding, fine-grained GPU acceleration, and memory-aware scheduling, all coordinated through a unified memory allocation mechanism that adheres to user-specified memory budgets. Evaluated across five Mac devices spanning an 8× range in NPU compute and a 32× range in memory capacity, the system consistently achieves efficient background indexing and real-time interactive search performance.
📝 Abstract
A search engine that embeds text, code, documents, images, audio and video into the same representation space has to run its encoder and keep its index somewhere, and almost every component built for the purpose assumes a server. We present omni-macos, which runs that whole engine, encoder, index and store, on the Mac the files are already on, so no file, query or vector ever leaves the machine. It keeps a background indexer and an interactive search box inside one memory budget the user sets: it re-encodes only the chunks an edit changes, hands the GPU smaller units while the user is typing, answers queries from a quantized replica with exact rescoring, and propagates that budget to the allocators that draw on unified memory. We measure every mechanism on five Macs spanning an eightfold range of accelerator width and a thirty-twofold range of memory, each one indexing its own local files.
Innovation
Methods, ideas, or system contributions that make the work stand out.
on-device search
omni-modal embedding
Apple Silicon
unified memory management
incremental indexing
🔎 Similar Papers
No similar papers found.
