DynImmune-BERT: Dynamic Immune Repertoire Modeling with Neural ODE Driven Continuous Transformers

2026-07-19

AI Summary

Existing static immune repertoire models struggle to capture the clonal dynamics of T cell receptor sequences—such as expansion, contraction, and reappearance—in longitudinal data. To address this limitation, this work proposes DynImmune-BERT, a continuous-time immune repertoire model that integrates neural ordinary differential equation (ODE)-driven clonal presence gating, bounded neighborhood self-attention, event-triggered state resets, and low-rank meta-adapters, initialized with deep adaptive centered log-ratio embeddings. The model jointly optimizes representations of both dominant and rare clones through a hybrid transport objective, without increasing parameter count as clone numbers grow. Experiments demonstrate that DynImmune-BERT significantly outperforms static baselines on longitudinal immune repertoire data, while enabling uncertainty quantification, calibration diagnostics, and clonal trajectory visualization—highlighting the need for cautious interpretation in cross-protocol analyses.

Abstract

Longitudinal T cell receptor repertoires contain signals of clonal expansion, contraction, disappearance, and reappearance after immune perturbation. Static repertoire language models usually summarize a sample as a bag of sequences, so the sampling interval, sequencing depth, and clone presence pattern are only weakly represented. This paper presents DynImmune-BERT, a continuous time repertoire model for patient level immune status prediction. The method combines depth adaptive centered log ratio initialization, clone presence gated Neural ordinary differential equation dynamics, bounded neighborhood self attention, event based state restart, and a hybrid transport objective that supervises dominant and rare clone mass. A low rank meta adapter initializes reappearing clonotypes while keeping the parameter count independent of the number of observed clones. The evaluation separates literature reported baselines from internally controlled temporal comparisons, reports uncertainty for small external cohorts, adds calibration and threshold diagnostics, and visualizes latent clone trajectories and attention neighborhoods. The results indicate that event aware temporal modeling can complement strong static encoders when longitudinal repertoire structure is available, while small external cohorts and protocol differences require cautious interpretation.

Problem

• immune repertoire
• longitudinal modeling
• clonal dynamics
• T cell receptor
• temporal immune response

Innovation

• Neural ODE
• continuous-time modeling
• immune repertoire
• Transformer
• clonal dynamics