Enhancing Smart Environments with Context-Aware Chatbots using Large Language Models

2025-02-20

AI Summary

This work addresses the limitation of conventional chatbots in perceiving environmental context and user behavior. We propose a context-aware, large language model (LLM)-enhanced framework tailored for intelligent environments. Methodologically, we introduce the first deep integration of centimeter-accurate ultra-wideband (UWB) localization, multimodal sensor fusion, and real-time human activity recognition (HAR) to construct a dynamic environment–behavior joint context; this context is then injected into an LLM via structured prompt engineering to enable situationally adaptive dialogue generation. Our key contribution is establishing a closed-loop paradigm—“physical perception → semantic understanding → language generation”—overcoming the constraints of static question-answering systems. Experiments on a real-world smart home dataset demonstrate a 42% improvement in dialogue relevance and a 37% increase in task completion rate, significantly enhancing the naturalness and practical utility of human–AI interaction.

Technology Category

• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Machine Learning: Large Multimodal Models (LMMs)
• Humans and AI: Human-Aware Planning and Behavior Prediction

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Personalized, context-aware and across-device search

Abstract

This work presents a novel architecture for context-aware interactions within smart environments, leveraging Large Language Models (LLMs) to enhance user experiences. Our system integrates user location data obtained through UWB tags and sensor-equipped smart homes with real-time human activity recognition (HAR) to provide a comprehensive understanding of user context. This contextual information is then fed to an LLM-powered chatbot, enabling it to generate personalised interactions and recommendations based on the user's current activity and environment. This approach moves beyond traditional static chatbot interactions by dynamically adapting to the user's real-time situation. A case study conducted from a real-world dataset demonstrates the feasibility and effectiveness of our proposed architecture, showcasing its potential to create more intuitive and helpful interactions within smart homes. The results highlight the significant benefits of integrating LLM with real-time activity and location data to deliver personalised and contextually relevant user experiences.

Problem

• Enhance smart environments with context-aware chatbots
• Integrate real-time activity and location data
• Deliver personalized, contextually relevant user experiences

Innovation

• LLM-powered context-aware chatbots
• UWB tags for user location
• Real-time human activity recognition