DynImmune-BERT: Dynamic Immune Repertoire Modeling with Neural ODE Driven Continuous Transformers

2026-07-19 · 13:25 UTC ·Rong Fu, Yongtai Liu, Xiaowen Ma... ·1 min read

Brief

DynImmune-BERT addresses longitudinal T cell‑repertoire dynamics by modeling patient‑level immune status in continuous time using Neural ODEs plus event-aware mechanisms. The approach combines specialized initialization, gated ODE clone dynamics, neighborhood self‑attention, and a hybrid transport loss to capture dominant and rare clone behavior. Results show temporal/event‑aware modeling complements static encoders; evaluation emphasizes uncertainty, calibration, and sensitivity to small external cohorts and protocol variation.

Why it matters

DynImmune-BERT (Rong Fu et al., arXiv:2607.17244v1, 2026-07-19) is a continuous-time T cell‑repertoire model that integrates depth-adaptive centered log-ratio initialization, clone-presence–gated Neural ODE dynamics, bounded neighborhood self-attention, event-based state restarts, and a hybrid transport objective supervising both dominant and rare clone mass.

Key details

Evaluation protocol separates literature baselines from internal temporal comparisons, reports uncertainty for small external cohorts, includes calibration and threshold diagnostics, visualizes latent clone trajectories and attention neighborhoods, and concludes event-aware temporal modeling can complement strong static encoders while small cohorts and protocol differences warrant cautious interpretation.

Abstract

Longitudinal T cell receptor repertoires contain signals of clonal expansion, contraction, disappearance, and reappearance after immune perturbation. Static repertoire language models usually summarize a sample as a bag of sequences, so the sampling interval, sequencing depth, and clone presence pattern are only weakly represented. This paper presents DynImmune-BERT, a continuous time repertoire model for patient level immune status prediction. The method combines depth adaptive centered log ratio initialization, clone presence gated Neural ordinary differential equation dynamics, bounded neighborhood self attention, event based state restart, and a hybrid transport objective that supervises dominant and rare clone mass. A low rank meta adapter initializes reappearing clonotypes while keeping the parameter count independent of the number of observed clones. The evaluation separates literature reported baselines from internally controlled temporal comparisons, reports uncertainty for small external cohorts, adds calibration and threshold diagnostics, and visualizes latent clone trajectories and attention neighborhoods. The results indicate that event aware temporal modeling can complement strong static encoders when longitudinal repertoire structure is available, while small external cohorts and protocol differences require cautious interpretation.