Privacy-Preserving LLM Interaction with Socratic Chain-of-Thought Reasoning and Homomorphically Encrypted Vector Databases

2025-06-19

AI Summary

To mitigate privacy risks arising from uploading sensitive personal data (e.g., emails, calendars, medical records) to the cloud when deploying LLMs as personal agents, this paper proposes a phased hybrid privacy-preserving architecture. First, a powerful cloud-based model (GPT-4o) generates data-agnostic Socratic Chain-of-Thought (CoT) subqueries. Second, private data is retrieved locally in milliseconds via a fully homomorphic encryption (FHE)-enabled vector database. Third, a lightweight on-device model (Llama-3.2-1B) synthesizes the final response. We introduce the Socratic CoT reasoning paradigm synergized with FHE-based vector retrieval—the first framework enabling encrypted semantic search over million-scale personal documents. Evaluated on the LoCoMo long-context QA benchmark, our hybrid approach achieves a 7.1% accuracy gain over standalone GPT-4o, with end-to-end latency under one second, thereby reconciling strong reasoning capability with strict on-device data confinement.

Technology Category

Machine Learning: PrivacySearch and Optimization: Learning to SearchNatural Language Processing: Safety and Robustness

Application Category

Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved informationSemantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactionsUser Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Large language models (LLMs) are increasingly used as personal agents, accessing sensitive user data such as calendars, emails, and medical records. Users currently face a trade-off: They can send private records, many of which are stored in remote databases, to powerful but untrusted LLM providers, increasing their exposure risk. Alternatively, they can run less powerful models locally on trusted devices. We bridge this gap. Our Socratic Chain-of-Thought Reasoning first sends a generic, non-private user query to a powerful, untrusted LLM, which generates a Chain-of-Thought (CoT) prompt and detailed sub-queries without accessing user data. Next, we embed these sub-queries and perform encrypted sub-second semantic search using our Homomorphically Encrypted Vector Database across one million entries of a single user's private data. This represents a realistic scale of personal documents, emails, and records accumulated over years of digital activity. Finally, we feed the CoT prompt and the decrypted records to a local language model and generate the final response. On the LoCoMo long-context QA benchmark, our hybrid framework, combining GPT-4o with a local Llama-3.2-1B model, outperforms using GPT-4o alone by up to 7.1 percentage points. This demonstrates a first step toward systems where tasks are decomposed and split between untrusted strong LLMs and weak local ones, preserving user privacy.

Problem

• Enables private LLM interactions without exposing sensitive user data
• Balances powerful untrusted LLMs with secure local model processing
• Achieves accurate responses while maintaining user data encryption

Innovation

• Socratic Chain-of-Thought Reasoning for privacy
• Homomorphically Encrypted Vector Database search
• Hybrid untrusted-local LLM task decomposition