Graph Masked Autoencoder for Spatio-Temporal Graph Learning

2024-10-14

arXiv.org

AI Summary

To address poor robustness in regional representation caused by strong noise and sparse labels in urban spatiotemporal graph data, this paper introduces the Spatiotemporal Heterogeneous Graph Neural Encoder (ST-HGAE), the first application of masked autoencoding to spatiotemporal graph learning. ST-HGAE jointly masks node features and graph structure, enabling generative self-supervised learning to automatically distill dynamic spatiotemporal dependencies. Its core innovations include a structure-aware masking strategy tailored for heterogeneous spatiotemporal graphs, and a dual reconstruction objective integrating node-level feature recovery with topology reconstruction. Evaluated on traffic flow, pedestrian flow, and crime prediction tasks, ST-HGAE consistently outperforms state-of-the-art methods—particularly under high noise levels and low label rates—while significantly enhancing modeling of dynamic spatial correlations among regions.

Technology Category

Machine Learning: Deep Generative Models & AutoencodersPlanning, Routing, and Scheduling: Optimization of Spatio-temporal SystemsKnowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning

Application Category

Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphsSearch and Retrieval-Augmented AI: Web query analysis, representation and understandingSemantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Effective spatio-temporal prediction frameworks play a crucial role in urban sensing applications, including traffic analysis, human mobility behavior modeling, and citywide crime prediction. However, the presence of data noise and label sparsity in spatio-temporal data presents significant challenges for existing neural network models in learning effective and robust region representations. To address these challenges, we propose a novel spatio-temporal graph masked autoencoder paradigm that explores generative self-supervised learning for effective spatio-temporal data augmentation. Our proposed framework introduces a spatial-temporal heterogeneous graph neural encoder that captures region-wise dependencies from heterogeneous data sources, enabling the modeling of diverse spatial dependencies. In our spatio-temporal self-supervised learning paradigm, we incorporate a masked autoencoding mechanism on node representations and structures. This mechanism automatically distills heterogeneous spatio-temporal dependencies across regions over time, enhancing the learning process of dynamic region-wise spatial correlations. To validate the effectiveness of our STGMAE framework, we conduct extensive experiments on various spatio-temporal mining tasks. We compare our approach against state-of-the-art baselines. The results of these evaluations demonstrate the superiority of our proposed framework in terms of performance and its ability to address the challenges of spatial and temporal data noise and sparsity in practical urban sensing scenarios.

Problem

• Overcoming noisy sparse spatial-temporal urban data limitations
• Learning meaningful region representations in spatial-temporal graphs
• Handling real-world urban data noise and sparsity challenges

Innovation

• Heterogeneous graph autoencoder for urban data
• Generative self-supervised learning framework
• Masked autoencoder for spatial-temporal patterns