nextlocllm: next location prediction using LLMs

2024-10-11

arXiv.org

AI Summary

Existing approaches model next-location prediction as discrete ID classification, neglecting spatial continuity and cross-city generalization. This paper introduces the first large language model (LLM)-based framework for cross-city human mobility modeling. First, it replaces categorical location IDs with normalized geographic coordinates to explicitly encode spatial continuity and enable zero-shot transfer across cities. Second, it leverages LLMs to encode natural-language descriptions of points of interest (POIs), jointly capturing functional semantics and trajectory context; efficient fine-tuning is achieved via partial parameter freezing and prompt prefix tuning. Third, it incorporates a retrieval-augmented decoding module to enforce spatial structural consistency in predictions. Extensive experiments demonstrate significant improvements over state-of-the-art methods under both supervised and zero-shot settings. The framework achieves strong cross-city generalization and high-precision localization on multiple unseen cities.

Technology Category

• Natural Language Processing: (Large) Language Models
• Machine Learning: Large Multimodal Models (LMMs)
• Planning, Routing, and Scheduling: Planning with Language Models

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Large language models for search

Abstract

Next location prediction is a critical task in human mobility analysis and serves as a foundation for various downstream applications. Existing methods typically rely on discrete IDs to represent locations, which inherently overlook spatial relationships and cannot generalize across cities. In this paper, we propose NextLocLLM, which leverages the advantages of large language models (LLMs) in processing natural language descriptions and their strong generalization capabilities for next location prediction. Specifically, instead of using IDs, NextLocLLM encodes locations based on continuous spatial coordinates to better model spatial relationships. These coordinates are further normalized to enable robust cross-city generalization. Another highlight of NextlocLLM is its LLM-enhanced POI embeddings. It utilizes LLMs' ability to encode each POI category's natural language description into embeddings. These embeddings are then integrated via nonlinear projections to form this LLM-enhanced POI embeddings, effectively capturing locations' functional attributes. Furthermore, task and data prompt prefix, together with trajectory embeddings, are incorporated as input for partly-frozen LLM backbone. NextLocLLM further introduces prediction retrieval module to ensure structural consistency in prediction. Experiments show that NextLocLLM outperforms existing models in next location prediction, excelling in both supervised and zero-shot settings.

Problem

• Predicting next location via coordinate regression instead of classification
• Modeling location semantics using LLM-enhanced POI embeddings
• Enabling cross-city generalization through unified semantic representation

Innovation

• Reformulates location prediction as coordinate regression
• Uses LLMs to extract functional semantics from POI descriptions
• Combines semantic embeddings with spatiotemporal trajectory representations