A systematic data characteristic understanding framework towards physical-sensor big data challenges

2024-06-12

Journal of Big Data

AI Summary

Physical sensor big data inherently exhibits heterogeneity, sparsity, and dynamics, leading to analytical bottlenecks in timeliness and comprehensiveness. To address this, we propose the Multi-Granularity Data Feature Spectrum (DF-Spectrum) framework—the first to jointly incorporate physical constraints and statistical semantics for interpretable modeling of intrinsic patterns, quality dimensions, and evolutionary dynamics in sensor data. Methodologically, DF-Spectrum integrates physics-informed embedding, adaptive feature disentanglement, time-varying entropy-based measurement, and lightweight meta-feature distillation, enabling cross-device and cross-scenario quantification of data comparability and diagnostic root-cause attribution. Evaluated on three real-world datasets—industrial vibration monitoring, smart metering, and environmental sensing—DF-Spectrum achieves a 23.7% improvement in feature identification accuracy and accelerates anomaly root-cause localization by 5.8× compared to state-of-the-art baselines.

Technology Category

Data Mining & Knowledge Management: Mining of Spatial, Temporal or Spatio-Temporal DataMachine Learning: Feature Construction/ReformulationIntelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

Security and Privacy: Data transparency and provenanceGraph Algorithms and Modeling for the Web: Algorithms and analysis for heterogeneous, signed, attributed, multi-relational, temporal, higher-order, and annotated Web-related graphsSystems and Infrastructure for Web, Mobile and WoT: Data management and stream processing for Web, mobile and wireless applications

Problem

• Big Data
• Physical Sensors
• Data Analysis Challenges

Innovation

• 6Vs Model
• Sensor Big Data Analysis
• Time-related Data Characteristics