Self-Supervised Representation Learning: From Spectral Foundation Models to Auroral Emission Spectra

2026-09-25

AI Summary

This study addresses the scarcity of annotated auroral spectra and the challenge of leveraging abundant unlabeled data by proposing a self-supervised pretraining framework based on a 1D Vision Transformer and a masked autoencoder. Without requiring annotations, this approach effectively reconstructs physically diagnostic features, while linear probing achieves classification performance comparable to expert-engineered features. Experimental results demonstrate that the fine-tuned model attains a Macro-AP of 88.5%, surpassing models trained from scratch on the full dataset using only 10% of labeled data. By significantly outperforming existing baselines, this work establishes an efficient paradigm for auroral spectral analysis in low-resource scenarios.

Abstract

Auroral spectrographs such as the Auroral Spectrograph In Skibotn (ASIS) record hundreds of thousands of emission spectra, but only a few hundred can be labelled by an expert. To exploit the rest, we pretrain a 1D Vision Transformer with a masked autoencoder on 223,000 unlabelled spectra. Without labels, its representation recovers the emission-line intensity ratios that physicists use to diagnose the precipitating particles (R^2 0.91 vs. 0.77 for an untrained control) and, under one linear probe, classifies as well as 13 features designed by experts. Fine-tuned, the model outperforms the previous supervised auroral classifier on its own benchmark (macro-AP 88.5 vs. 77.8), reaches 0.870 mAP, and exceeds the same architecture trained from scratch by +0.159 with 10% of the labels; attribution shows that it uses both N2+ bands. Could an existing pretrained model replace it? Two astronomical spectral foundation models and a time-series model transfer according to their spectral window: SpectraFM, trained in the infrared, falls below the untrained control, whereas SpecFormer, trained in the optical, approaches in-domain pretraining without reaching it.

Problem

• auroral emission spectra
• self-supervised representation learning
• label scarcity
• foundation models
• spectral classification

Innovation

• Self-Supervised Learning
• Masked Autoencoder
• Vision Transformer
• Auroral Emission Spectra
• Foundation Models