RAFM: Retrieval-Augmented Flow Matching for Unpaired CBCT-to-CT Translation

2026-02-28

AI Summary

This work addresses the challenges in radiation therapy planning posed by cone-beam computed tomography (CBCT) images, which suffer from severe artifacts and unreliable Hounsfield Unit (HU) values that preclude their direct use for dose calculation, compounded by the absence of real paired CBCT–CT datasets. To overcome these limitations, the authors propose a Retrieval-Augmented Flow Matching (RAFM) framework that integrates rectified flow with a frozen DINOv3 encoder. By constructing a global CT memory bank, RAFM enables semantically consistent pseudo-pairing, thereby enhancing distribution-level alignment between unpaired CBCT and CT images and mitigating training instability under limited medical data. Evaluated under the strict unpaired protocol of SynthRAD2023, RAFM significantly outperforms existing methods across multiple metrics, including FID, MAE, SSIM, PSNR, and SegScore.

Technology Category

• Computer Vision: Medical and Biological Imaging
• Machine Learning: Calibration & Uncertainty Quantification
• Planning, Routing, and Scheduling: Activity and Plan Recognition

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

Cone-beam CT (CBCT) is routinely acquired in radiotherapy but suffers from severe artifacts and unreliable Hounsfield Unit (HU) values, limiting its direct use for dose calculation. Synthetic CT (sCT) generation from CBCT is therefore an important task, yet paired CBCT--CT data are often unavailable or unreliable due to temporal gaps, anatomical variation, and registration errors. In this work, we introduce rectified flow (RF) into unpaired CBCT-to-CT translation in medical imaging. Although RF is theoretically compatible with unpaired learning through distribution-level coupling and deterministic transport, its practical effectiveness under small medical datasets and limited batch sizes remains underexplored. Direct application with random or batch-local pseudo pairing can produce unstable supervision due to semantically mismatched endpoint samples. To address this challenge, we propose Retrieval-Augmented Flow Matching (RAFM), which adapts RF to the medical setting by constructing retrieval-guided pseudo pairs using a frozen DINOv3 encoder and a global CT memory bank. This strategy improves empirical coupling quality and stabilizes unpaired flow-based training. Experiments on SynthRAD2023 under a strict subject-level true-unpaired protocol show that RAFM outperforms existing methods across FID, MAE, SSIM, PSNR, and SegScore. The code is available at https://github.com/HiLab-git/RAFM.git.

Problem

• CBCT-to-CT translation
• unpaired image translation
• synthetic CT generation
• medical image artifacts
• Hounsfield Unit calibration

Innovation

• Rectified Flow
• Unpaired Translation
• Retrieval-Augmented
• CBCT-to-CT Synthesis
• Medical Image Generation