Refining Cytology Predictions with Conditional Random Fields

2026-09-25

AI Summary

This study addresses the limited zero-shot classification performance of vision-language models (VLMs) on cytological images and the poor cross-staining generalizability of existing conditional random fields (CRFs). We propose CytoCRF, which adapts pairwise potentials to staining-specific chromatin features and incorporates a multi-backbone fusion strategy to enrich neighborhood information for refining VLM predictions. To our knowledge, this work is the first to adapt CRFs to cytological data across diverse staining protocols, revealing that neighborhood topology is more critical than pairwise potential computation. Evaluated on ten datasets, CytoCRF outperforms existing baselines by up to 13.6% and achieves a 33.7% improvement over zero-shot performance using only fifty labeled samples.

Abstract

Vision-language models (VLMs) achieve strong zero-shot (ZS) classification on histology images but do not perform as well on cytology, whose stains and cell morphology differ markedly compared to histology. Conditional random fields (CRFs) can refine noisy VLM predictions by propagating information across patches, but existing CRF frameworks were designed for histopathology and do not transfer to cytology datasets, released as independent patch pools spanning multiple staining protocols. We introduce CytoCRF, which adapts the pairwise terms to cytology by targeting chromatin and cytology-specific staining, and further enrich the neighborhood of each potential term by combining multiple backbones. Across ten cytology datasets, CytoCRF outperforms existing CRF frameworks at every annotation budget, reaching +13.6 percentage points over the best baseline and +33.7 over ZS with only 50 annotations. Combining information from multiple backbones brings further gains, showing that the neighborhood topology matters more than the pairwise potential computed over it.

Problem

• Vision-language models
• Cytology classification
• Conditional random fields
• Zero-shot learning

Innovation

• Conditional Random Fields
• Vision-Language Models
• Cytology
• Multi-backbone Fusion
• Zero-shot Classification