Stitching and dimensionality effects on large artificially generated volume datasets

2026-06-18

AI Summary

This study addresses the challenge of artifacts introduced by tiling strategies when generating large-scale images with deep learning, particularly in cryo-electron microscopy data. Using CycleGAN, the authors systematically evaluate three tiling approaches combined with 2D and 3D patching schemes in terms of image generation quality and their impact on downstream mitochondrial segmentation performance. They find that commonly used perceptual metrics such as FID fail to capture subtle artifacts that significantly degrade segmentation accuracy. To mitigate this, the work proposes an orthogonal-direction ensemble prediction strategy that effectively enhances generation quality for low-quality volumetric data. Experimental results show that 3D models yield marginally better results than 2D models under artifact-free tiling but incur substantially higher computational costs; in contrast, 2D models enable larger batch sizes and more stable training. This work highlights a critical disconnect between standard evaluation metrics and task-specific performance in biomedical image domain adaptation.

Abstract

Generating large images via deep learning requires patching input data to accommodate hardware memory limitations, then assembling output patches, a process that can introduce stitching artifacts when neighboring patches do not align at borders. While these artifacts are known to affect segmentation tasks, their impact on generative models for style-transfer remains poorly understood. We investigated three stitching approaches and two patch dimensionalities (2D vs 3D) using cycleGAN models trained on cryo-electron microscopy datasets. We evaluated both perceptual quality and performance on downstream mitochondria segmentation. Our key findings reveal that: (1) FID scores fail to detect subtle stitching artifacts that significantly impact downstream segmentation performance, (2) 3D models with artifact-free stitching marginally outperform 2D models on downstream tasks, though the improvement barely justifies the computational cost, and (3) 2D models train more stably due to larger batch sizes. Additionally, we demonstrate that ensembling predictions from three orthogonal directions can improve low-quality volumes but provides no benefit for high-quality outputs. These results demonstrate that maximizing generative model performance on large scientific datasets requires careful consideration and mitigation of stitching artifacts, and that perceptual metrics alone are insufficient for evaluating domain adaptation quality in biomedical imaging.

Problem

• stitching artifacts
• generative models
• style-transfer
• dimensionality
• biomedical imaging

Innovation

• stitching artifacts
• 3D vs 2D generative models
• CycleGAN
• domain adaptation
• biomedical image synthesis