Group Equivariant Diffusion for Anomaly Detection in Computational Cytology

2026-07-28

AI Summary

This study addresses the challenges of unsupervised anomaly detection in whole-slide cytopathology images, where malignant cells are rare, highly heterogeneous, and sparsely annotated. Existing methods often neglect rotational and reflectional symmetries, leading to inconsistent reconstructions and unstable anomaly scores. To overcome this, the work introduces D4 group equivariance into diffusion models for the first time, proposing a D4-equivariant U-Net architecture. Coupled with equivariant noise coupling at inference—implemented via optional frame averaging—it enables transformation-invariant pseudo-healthy reconstructions without requiring additional annotations. Evaluated on bone marrow and peripheral blood smear datasets, the method significantly improves anomaly detection AUC and Top-K recall while substantially reducing score variance under geometric transformations, outperforming non-equivariant generative models, deep one-class classification, and multiple instance learning baselines.

Abstract

Computational cytology on whole-slide images is challenging because malignant cells are rare, heterogeneous, and annotated slides are scarce. Anomaly detection frameworks can be trained on normal slide-negative patches and then applied at test time to flag abnormal patches in held-out slides. Most unsupervised anomaly detection approaches including generative ones (GAN-based and diffusion-based), are tuned to organ-level imaging and require large curated datasets. In cytology the signal is cell-centric: rotating or flipping a single-cell patch does not change its diagnostic class, yet standard diffusion models treat transformed views as distinct inputs, leading to transformation-dependent reconstructions and unstable anomaly scores. We propose a D4-equivariant diffusion framework that enforces rotation and reflection symmetry both architecturally, via a D4-equivariant U-Net, and at inference, via equivariant noise coupling and (optionally) frame averaging. This alignment with biological invariance yields transformation-consistent pseudo-healthy reconstructions and more stable anomaly ranking under symmetry. On two publicly available cytology datasets of bone marrow and peripheral blood smears, our D4-equivariant diffusion models achieve higher AUC and retrieve more abnormal cells in the top K predictions than non-equivariant generative baselines, a deep one-class, and a multiple instance learning based method, while substantially reducing score variance across rotations and flips. Code is available at https://swchmida.github.io/D4diffCyto/.

Problem

• anomaly detection
• computational cytology
• group equivariance
• diffusion models
• symmetry invariance

Innovation

• Group Equivariance
• Diffusion Models
• Anomaly Detection
• Computational Cytology
• D4 Symmetry