Map as a By-product: Collective Landmark Mapping from IMU Data and User-provided Texts in Situated Tasks

2025-09-03

AI Summary

Indoor semantic mapping remains challenging due to the high cost and impracticality of dedicated surveying. Method: This paper proposes a crowdsourced landmark mapping approach that requires no specialized instrumentation. It leverages smartphone IMU trajectories and free-text notes collected implicitly during routine navigation and annotation tasks, automatically detecting, aligning, and aggregating multi-user observations of indoor landmarks. The method jointly models spatial and semantic information by integrating inertial trajectory analysis with natural language processing, embodying a “map-as-byproduct” paradigm—where mapping is seamlessly embedded within authentic user workflows. Results: Evaluated in retail restocking and office inspection scenarios, the generated semantic landmark maps (with precise locations and functional labels) achieve superior landmark coverage and semantic consistency compared to existing crowdsourced mapping methods. This work significantly expands the applicability boundary of lightweight, sustainable indoor semantic mapping.

Technology Category

• Intelligent Robots: Localization, Mapping, and Navigation
• Humans and AI: Crowd Sourcing and Human Computation
• Natural Language Processing: Language Grounding & Multi-modal NLP

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Economics, Online Markets and Human Computation: Architectures and workflows that use LLMs for crowd work
• User Modeling, Personalization and Recommendation: Practical large-scale studies of user experience

Abstract

This paper presents Collective Landmark Mapper, a novel map-as-a-by-product system for generating semantic landmark maps of indoor environments. Consider users engaged in situated tasks that require them to navigate these environments and regularly take notes on their smartphones. Collective Landmark Mapper exploits the smartphone's IMU data and the user's free text input during these tasks to identify a set of landmarks encountered by the user. The identified landmarks are then aggregated across multiple users to generate a unified map representing the positions and semantic information of all landmarks. In developing the proposed system, we focused specifically on retail applications and conducted a formative interview with stakeholders to confirm their practical needs that motivate the map-as-a-byproduct approach. Our user study demonstrates the feasibility of the proposed system and its superior mapping performance in two different setups: creating a product availability map from restocking checklist tasks at a retail store and constructing a room usage map from office inspection tasks, further demonstrating the potential applicability to non-retail applications.

Problem

• Generating semantic landmark maps from IMU data and user text
• Aggregating landmarks across multiple users for unified mapping
• Creating indoor environment maps during situated navigation tasks

Innovation

• Generates semantic maps from IMU and text data
• Aggregates landmarks across multiple users collectively
• Uses smartphone inputs during situated navigation tasks