OmniSpectra: A Unified Foundation Model for Native Resolution Astronomical Spectra

2026-01-21

AI Summary

This work proposes the first unified foundation model capable of handling multi-survey, multi-configuration astronomical spectra of arbitrary length at native resolution, overcoming the limitations of existing models that are constrained to fixed wavelength ranges and specific instruments. The architecture integrates adaptive chunking, sinusoidal global wavelength encoding, deep convolutional local positional embeddings, and a validity-aware self-attention mechanism to enable general-purpose representation learning across diverse tasks. Evaluated in a zero-shot setting, the model significantly outperforms task-specific counterparts and achieves state-of-the-art performance in stellar classification, redshift estimation, and galaxy stellar property prediction.

Technology Category

• Machine Learning: Deep Neural Architectures and Foundation Models
• Computer Vision: Multi-modal Vision
• Natural Language Processing: Language Grounding & Multi-modal NLP

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Economics, Online Markets and Human Computation: Architectures and workflows that use LLMs for crowd work

Abstract

We present OmniSpectra, the first native-resolution foundation model for astronomy spectra. Unlike traditional models, which are limited to fixed-length input sizes or configurations, OmniSpectra handles spectra of any length at their original size, without resampling or interpolation. Despite the large-scale spectroscopic data from diverse surveys fueling the rapid growth of astronomy, existing foundation models are limited to a fixed wavelength range and specific instruments. OmniSpectra is the first foundation model to learn simultaneously from multiple real-world spectra surveys with different configurations at a large scale. We achieve this by designing a novel architecture with adaptive patching across variable lengths, sinusoidal global wavelength encoding, local positional embeddings through depthwise convolution, and validity-aware self-attention masks. Allowing us to learn multi-scale spatial patterns while skipping attention for invalid patches. Even with a limited training example, OmniSpectra demonstrates excellent zero-shot generalization compared to methods tailored for specific tasks. This transfer learning capability makes this model the state-of-the-art across various astronomy tasks, including source classification, redshift estimation, and properties prediction for stars and galaxies. OmniSpectra reduces the need for training individual models for different tasks from scratch, establishing itself as the next-generation astronomy foundation model.

Problem

• astronomical spectra
• foundation model
• native resolution
• variable-length input
• multi-survey learning

Innovation

• foundation model
• native-resolution spectra
• adaptive patching
• wavelength encoding
• zero-shot generalization