Towards Privacy-Preserving and Personalized Smart Homes via Tailored Small Language Models

2025-07-10

AI Summary

Existing LLM-based smart home assistants require uploading user commands, profiles, and device configurations to remote servers, posing severe privacy risks. To address this, we propose HomeLLaMA–PrivShield—a novel collaborative framework that integrates a lightweight on-device small language model (SLM) with an optional differentially private remote service, enabling personalized understanding and proactive interaction entirely at the edge. Our approach synergistically combines knowledge distillation, on-device continual learning, differential privacy mechanisms, and fine-grained user configuration management to build an end-to-end privacy-first system. Evaluated on our custom-built benchmark DevFinder, the framework achieves state-of-the-art performance. A user study (N=100) demonstrates that, while strictly ensuring data remains within the local domain, the system significantly improves response quality, personalization capability, and users’ trust in privacy preservation.

Technology Category

• Machine Learning: Privacy
• Humans and AI: Human-in-the-loop Machine Learning
• Natural Language Processing: Safety and Robustness

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Security and Privacy: Privacy-enhancing technologies
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information

Abstract

Large Language Models (LLMs) have showcased remarkable generalizability in language comprehension and hold significant potential to revolutionize human-computer interaction in smart homes. Existing LLM-based smart home assistants typically transmit user commands, along with user profiles and home configurations, to remote servers to obtain personalized services. However, users are increasingly concerned about the potential privacy leaks to the remote servers. To address this issue, we develop HomeLLaMA, an on-device assistant for privacy-preserving and personalized smart home serving with a tailored small language model (SLM). HomeLLaMA learns from cloud LLMs to deliver satisfactory responses and enable user-friendly interactions. Once deployed, HomeLLaMA facilitates proactive interactions by continuously updating local SLMs and user profiles. To further enhance user experience while protecting their privacy, we develop PrivShield to offer an optional privacy-preserving LLM-based smart home serving for those users, who are unsatisfied with local responses and willing to send less-sensitive queries to remote servers. For evaluation, we build a comprehensive benchmark DevFinder to assess the service quality. Extensive experiments and user studies (M=100) demonstrate that HomeLLaMA can provide personalized services while significantly enhancing user privacy.

Problem

• Privacy leaks in remote server-based smart home assistants
• Balancing personalization and privacy in smart home interactions
• On-device small language models for secure smart home services

Innovation

• On-device SLM for privacy-preserving smart homes
• Continuous local SLM and profile updates
• Optional privacy-protected remote query service