extract time-invariant features

Time-series forecasting suffers from poor generalization and low efficiency due to tight coupling among sequence representation, information extraction, and future projection. To address this, we propose a modular forecasting framework that decouples the pipeline into three independently optimizable stages—representation learning, information extraction, and target projection—enabling flexible, task-aware component configuration. Our approach innovatively integrates convolutional layers with a lightweight self-attention mechanism, achieving efficient local feature modeling while capturing long-range temporal dependencies. Evaluated on seven benchmark datasets, the method consistently outperforms existing state-of-the-art models in prediction accuracy, while requiring significantly fewer parameters and achieving faster training and inference speeds. This demonstrates substantial improvements in both statistical performance and computational efficiency.

High-dimensional data analysis faces fundamental challenges including empirically unjustified embedding method selection, poorly characterized performance bounds, and fragmented theoretical debates. This study systematically reviews mainstream dimensionality reduction techniques—including t-SNE, UMAP, PCA, and autoencoders—synthesizing scattered literature and key controversies to propose the first practice-oriented three-dimensional framework for low-dimensional embeddings: “generation–evaluation–application.” We conduct a comprehensive empirical evaluation across diverse real-world datasets and downstream tasks, rigorously characterizing each algorithm’s trade-offs in preserving local versus global structure, robustness to noise and hyperparameter variation, and interpretability. Our analysis establishes clear applicability boundaries and inherent limitations for each method. The resulting best-practice guidelines integrate theoretical rigor with engineering feasibility, providing the field with standardized evaluation protocols and principled criteria for method selection. (149 words)

Time series exhibit structural characteristics—including change points, repetitive patterns, and local stationarity—yet prevailing representation learning methods often neglect their non-stationarity and heterogeneity, leading to inefficient and brittle long-sequence modeling. To address this, we propose a lightweight, unsupervised preprocessing framework: it first performs multi-scale statistical change-point detection guided by the Bayesian Information Criterion (BIC) for structure-aware segmentation; then applies sliding-window merging and mean aggregation to produce compact tokenized sequences; finally leverages Gaussian Mixture Models (GMM) to preserve semantic fidelity. The method requires no model coupling and is fully plug-and-play. Evaluated across 150+ benchmark datasets, it achieves up to 30× sequence compression while retaining 85–90% of original model performance—substantially outperforming uniform sampling and clustering baselines. It demonstrates superior efficiency, robustness, and scalability.

To address the challenges of complex feature fusion and poor scalability in unsupervised multimodal time-series representation learning, this paper proposes a spectral-graph-theoretic framework for implicit cross-modal alignment. Methodologically, it abandons explicit multimodal feature fusion and instead adopts a single-encoder, multi-view architecture: diverse time-series views—such as frequency-domain, image-based, and symbolic representations—are generated via modality-agnostic temporal transformations, and spectral-graph-guided contrastive loss enables unsupervised cross-modal alignment. This lightweight design implicitly captures latent inter-modal dependencies, strengthening inductive bias and generalization capacity. Extensive experiments across multiple domains demonstrate state-of-the-art performance on downstream tasks—including classification, forecasting, and anomaly detection—surpassing existing unsupervised methods by an average accuracy gain of 3.2%–7.8%.

This paper addresses unsupervised representation learning for sequential data. We propose a novel probabilistic flow decomposition framework that disentangles the latent-space dynamics into two orthogonal vector fields: a sparse curl-free field (corresponding to an irrotational potential field) and a divergence-free field (corresponding to a solenoidal rotational field), with sparsity priors newly imposed on both components. Within a variational autoencoder framework, our method jointly optimizes representation encoding, velocity field estimation, and field-structure inference, implicitly learning approximately equivariant representations. Compared to prior approaches, our model simultaneously achieves static representation disentanglement and independence of dynamic transformation primitives, yielding significant improvements in data likelihood and unsupervised equivariance error across multiple sequence transformation benchmarks—achieving state-of-the-art performance. Crucially, the learned vector fields admit clear physical interpretations grounded in classical vector calculus.

Existing Transformer-based time series forecasting models overemphasize temporal modeling, resulting in high computational overhead with marginal performance gains and severely constrained representation capacity due to suboptimal embedding design. To address this, we propose Hybrid Temporal–Multivariate Embedding (HTME), a lightweight yet effective embedding framework that jointly captures temporal dynamics and inter-variable dependencies via two synergistic components: a temporal feature extraction module and a multivariate relational modeling module. HTME is plug-and-play compatible with standard Transformer architectures and enhances sequence representation without increasing decoding complexity. Extensive experiments across eight real-world datasets demonstrate that HTME consistently outperforms state-of-the-art baselines—achieving an average 6.2% reduction in MAE and a 31% decrease in FLOPs—thereby validating the fundamental benefit of multivariate-aware embedding for time series modeling.

This paper addresses the challenges in online learning arising from dynamically growing and unbounded categorical vocabularies, as well as performance degradation of conventional deterministic embeddings due to order sensitivity and catastrophic forgetting. We propose a probabilistic hashing embedding framework that integrates randomized hashing with Bayesian online learning, ensuring order-invariance, fixed-parameter scalability, and adaptive vocabulary evolution. Our method employs an efficient, scalable incremental inference algorithm to update the posterior distributions of hash embeddings and other latent variables in real time. Extensive experiments across streaming classification, sequential modeling, and recommendation tasks demonstrate significant improvements over state-of-the-art approaches. The proposed method incurs only 2–4× the memory overhead of a one-hot embedding table, achieving both high accuracy and computational efficiency.

This study addresses the challenge of learning generalizable temporal representations from longitudinal electronic health records of patients with chronic kidney disease (CKD). The authors propose a time-aware LSTM (T-LSTM) to learn task-agnostic clinical embeddings from the MIMIC-IV database. Compared to standard LSTM and attention-augmented LSTM variants, T-LSTM achieves improved generalization while preserving structured temporal representations. Experimental results demonstrate that the learned embeddings yield a CKD staging classification accuracy of 0.74 (Davies–Bouldin index: 9.91) and significantly enhance downstream ICU mortality prediction performance, achieving accuracies of 0.82–0.83—outperforming end-to-end trained baselines. These findings validate the effectiveness of T-LSTM as a robust framework for generating universal clinical time-series representations.

Existing retrieval-augmented time series forecasting methods fail to distinguish between invariant structures and dynamic variations within sequences, leading to limited performance in trend-dominated or distribution-shift scenarios. This work proposes a retrieval-guided invariant–dynamic decomposition framework that, for the first time, leverages retrieved sequences as implicit environmental samples. Through attention-based aggregation and a routing mechanism, the model disentangles representations into invariant and dynamic components, which are separately predicted and subsequently fused. The approach enables invariant learning without requiring explicit environment labels and is theoretically shown to reduce variance and approximate invariant representations via retrieval aggregation. Experiments demonstrate that the proposed framework significantly outperforms current foundation models and retrieval baselines in zero-shot settings, substantially improving prediction accuracy and robustness under distribution shifts.

This study addresses the limited interpretability of existing deep clustering methods, whose latent clusters are difficult to associate with observable waveform characteristics. To this end, we propose WAVE, a framework that pioneers joint representation learning by treating time-series signals and their deterministically rendered waveform images as complementary views. By integrating contrastive learning with multimodal fusion strategies, WAVE achieves deep alignment between fine-grained temporal variations and holistic visual patterns, yielding highly discriminative embeddings that endow clustering results with intuitive, waveform-level traceability. Extensive experiments on ten public datasets demonstrate that WAVE attains state-of-the-art macro-averaged clustering performance and the best average ranking. Furthermore, case studies validate that the discovered cluster structures can be directly inspected through raw waveforms, thereby bridging the gap between latent representations and physically interpretable signal features.