Conditional Random Fields for Interactive Refinement of Histopathological Predictions

2026-01-17

AI Summary

Existing vision-language models exhibit limited accuracy in zero-shot cancer detection and staging on histopathology images. To address this, this work proposes HistoCRF, a novel framework that, for the first time, integrates a human-in-the-loop mechanism into a conditional random field (CRF)-based refinement pipeline for pathological images. By introducing an innovatively designed pairwise potential function that promotes label diversity, HistoCRF enables interactive refinement of zero-shot predictions through expert knowledge without requiring additional model training. Evaluated across five multi-organ histopathology datasets, the method improves average accuracy by 16.0% in the absence of annotations; with only 100 labeled samples, performance increases by 27.5%, and further rises to 32.6% when combined with human-in-the-loop feedback.

Technology Category

• Computer Vision: Medical and Biological Imaging
• Humans and AI: Human-in-the-loop Machine Learning
• Machine Learning: Multi-instance/Multi-view Learning

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Assisting pathologists in the analysis of histopathological images has high clinical value, as it supports cancer detection and staging. In this context, histology foundation models have recently emerged. Among them, Vision-Language Models (VLMs) provide strong yet imperfect zero-shot predictions. We propose to refine these predictions by adapting Conditional Random Fields (CRFs) to histopathological applications, requiring no additional model training. We present HistoCRF, a CRF-based framework, with a novel definition of the pairwise potential that promotes label diversity and leverages expert annotations. We consider three experiments: without annotations, with expert annotations, and with iterative human-in-the-loop annotations that progressively correct misclassified patches. Experiments on five patch-level classification datasets covering different organs and diseases demonstrate average accuracy gains of 16.0% without annotations and 27.5% with only 100 annotations, compared to zero-shot predictions. Moreover, integrating a human in the loop reaches a further gain of 32.6% with the same number of annotations. The code will be made available on https://github.com/tgodelaine/HistoCRF.

Problem

• Histopathological image analysis
• Zero-shot prediction refinement
• Interactive annotation
• Label accuracy improvement
• Human-in-the-loop

Innovation

• Conditional Random Fields
• Histopathology
• Vision-Language Models
• Human-in-the-loop
• Zero-shot Refinement