CAPT: A Multi-task Continuous Autoregressive Transformer enabling Cross-dataset and Cross-species Transfer for Calcium Population Dynamics

2026-07-25

AI Summary

Existing models struggle to generalize neural population dynamics across new datasets, experimental paradigms, or species. To address this limitation, this work proposes CAPT (Continuous Autoregressive Transformer), which directly models calcium imaging trajectories through continuous segment tokenization, enabling end-to-end pretraining and efficient transfer via lightweight adapter modules while keeping the backbone frozen. CAPT establishes the first unified pretrained model capable of generalizing across laboratories and species—including mice, larval zebrafish, and C. elegans—across diverse downstream tasks, while constructing a shared functional embedding space. Experiments demonstrate that CAPT significantly outperforms current baselines in neural activity prediction and behavior decoding; notably, in C. elegans NeuroPAL data, the learned embeddings effectively capture anatomical cell identity structure.

Abstract

Large-scale calcium imaging has created an opportunity to build foundation-style models for neural population dynamics, but a central question remains unresolved: \textbf{whether a model pretrained on one collection of recordings can generalize to new datasets, experimental paradigms, and even species.} Existing approaches are often designed for specific tasks and evaluated on a single dataset, making it unclear whether their learned representations are reusable for new calcium trace datasets. To tackle this gap, we present \textbf{CAPT}, a \textbf{C}ontinuous \textbf{A}utoregressive \textbf{P}opulation \textbf{T}ransformer for calcium population dynamics. CAPT models continuous calcium traces directly through a continuous patch tokenization strategy and is trained autoregressively, enabling end-to-end pretraining and adaptation to diverse downstream tasks. We first pretrain CAPT on a large-scale mouse calcium imaging dataset and evaluate its transferability across independent mouse, larval zebrafish, and \textit{C. elegans} datasets collected by different laboratories. In these transfer settings, the pretrained backbone is frozen and only adaptation modules are updated. Across neural population forecasting and behavior decoding tasks, CAPT consistently outperforms specialized and general-purpose baselines. Alongside predictive performance, multimodal analyses using NeuroPAL annotations in \textit{C. elegans} datasets show that CAPT embeddings form a shared functional space across datasets and capture anatomical cell-identity-related structure. These results suggest that the continuous autoregressive modeling opens up possibilities for a simple route towards general-purpose neural foundation models for calcium imaging, which can generalize across datasets, experimental paradigms, and species.

Problem

• cross-dataset generalization
• cross-species transfer
• calcium imaging
• neural population dynamics
• foundation models

Innovation

• continuous autoregressive modeling
• cross-species transfer
• calcium imaging foundation model
• population dynamics
• multimodal neural embedding