Beyond Foundation Models: Dimension-Aware Neural Architecture Search with Small-Data Representation Models for Cryocooler Lifetime Prediction

2026-08-07

AI Summary

This study addresses the challenge of equipment remaining useful life prediction under small-sample, multivariate telemetry data by proposing FSD-RM, a representation learning framework tailored for low-data regimes. The framework leverages unsupervised pretraining with a hybrid architecture combining 1D CNN, LSTM, GRU, and Transformer encoder modules, followed by a two-stage prediction pipeline. Its key innovation lies in introducing dimension-aware neural architecture search (NAS), which jointly optimizes model structure and input feature dimensions without requiring large-scale pretraining. Experiments on cryocooler telemetry data demonstrate that FSD-RM achieves competitive predictive performance while significantly reducing training cost and model complexity, and simultaneously enhancing model interpretability.

Abstract

Large-scale pretrained time-series models achieve strong results through large-scale pretraining and task-agnostic representation learning, but they rely on abundant, diverse data that industrial and scientific domains often lack. We therefore propose the FSD-RM (Family of Small-Data Representation Models) paradigm as a practical alternative for limited, domain-specific telemetry. Rather than relying on large-scale pretraining, we focus on capacity-controlled representation learning using established encoder architectures (CNN1D, LSTM, GRU, Transformer), selected for their suitability in small-data settings and interpretability. These encoders are trained unsupervised on multivariate telemetry data and integrated into a two-stage pipeline for downstream lifetime prediction. To systematically examine architectural trade-offs under data constraints, we employ \textbf{dimension-aware neural architecture search (NAS)} to jointly optimize model capacity and input dimensionality. Experiments on cryocooler telemetry show that the proposed approach achieves competitive predictive performance while reducing training cost and model complexity. The contribution lies in combining established representation learning techniques within a coherent, NAS-driven framework tailored to small-data regimes, with explicitly defined parameter settings and design choices. The results indicate that effective representation learning can be achieved without large-scale pretraining when appropriate inductive bias and capacity control are applied.

Problem

• small-data
• lifetime prediction
• representation learning
• cryocooler
• time-series

Innovation

• dimension-aware neural architecture search
• small-data representation learning
• cryocooler lifetime prediction
• capacity-controlled models
• unsupervised encoder