MUL-T: Decoding Spatial Cellular Architecture in Multiplexed Tissue Images

2026-07-30

AI Summary

This work addresses the limitations of existing methods in spatial transcriptomics image analysis, which rely on handcrafted features that generalize poorly across diverse marker panels and fail to effectively model cellular phenotypes and their spatial contextual relationships. The study introduces masked context prediction—a novel self-supervised learning strategy—into this domain, proposing a lightweight Transformer framework that learns contextualized [CLS] embeddings from discretized cell markers to capture high-order cellular interactions. Requiring no task-specific supervision, the method achieves strong performance with high computational efficiency. It significantly outperforms conventional feature-based approaches across multiple tasks, including tumor pattern classification, patient stratification, PD-L1 positivity prediction, and cross-dataset treatment response forecasting, matching the accuracy of large Vision Transformer (ViT) models while using substantially fewer parameters and lower training costs.

Abstract

Understanding tissue organisation in multiplexed imaging requires modelling both cellular phenotypes and their spatial context. Existing approaches typically rely on handcrafted features, such as marker intensity statistics or cell-type proportions, which often fail to scale or generalise across cohorts with heterogeneous marker panels. We introduce MUL-T, a lightweight transformer framework that reframes tissue architecture as a masked contextual prediction task over discrete cell tokens. By learning contextualised [CLS] embeddings without task-specific supervision, the model captures higher-order cellular interactions while remaining computationally efficient. We evaluate MUL-T on several clinically relevant downstream tasks, including core-level tumour pattern classification, patient-level grading, PD-L1 positivity prediction, and cross-dataset treatment response prediction. Across tasks, MUL-T consistently outperforms classical feature-based baselines and achieves performance comparable to a foundation ViT model, despite substantially fewer parameters and lower training cost.

Problem

• multiplexed tissue imaging
• spatial cellular architecture
• cellular phenotypes
• heterogeneous marker panels
• tissue organization

Innovation

• MUL-T
• transformer
• spatial cellular architecture
• multiplexed tissue imaging
• contextual prediction