fid computation

This work systematically investigates the discrepancy between Fréchet Inception Distance (FID) and human perception, demonstrating that low FID scores do not necessarily correspond to high-quality generated outputs. The study identifies the geometric structure of the reference dataset—particularly its distribution density and effective rank—as the primary cause of FID’s unreliability. Through empirical analyses involving precision-recall decomposition, multiple feature spaces, and ablation studies on distance metrics, the authors reveal that FID exhibits reasonable behavior on concentrated datasets but becomes misleading on dispersed ones. Experiments across six datasets confirm that the geometric properties of the reference data critically influence the fidelity of distribution-based evaluation metrics. The findings advocate for interpreting such metrics in conjunction with the underlying dataset geometry, offering a more reliable foundation for evaluating generative models.

This paper identifies a systematic misalignment between generic generative evaluation metrics—such as Fréchet Inception Distance (FID)—and downstream task performance (e.g., classification or segmentation) in retinal image synthesis. Method: Through systematic experiments across multimodal retinal datasets (fundus photography and OCT), we empirically analyze the correlation between FID (and its variants) and actual gains in downstream model performance. Contribution/Results: We provide the first empirical evidence that FID scores fail to predict whether synthetic data meaningfully improve downstream model accuracy. To address this, we propose a “task-driven evaluation” paradigm, advocating direct assessment via target downstream task performance—replacing proxy metrics reliant on ImageNet-pretrained features. Our findings are robustly replicated across multiple public retinal image benchmarks, offering both methodological insight and practical guidance for evaluating biomedical image generation.

This work addresses the limited reliability of the Fréchet Inception Distance (FID) in non-natural image domains such as medical imaging, where its dependence on an ImageNet1K-pretrained Inception-v3 model impedes effective representation of out-of-distribution data. To mitigate this issue, the authors propose generating stochastic embedding representations via Monte Carlo Dropout and computing the predictive variance of FID feature embeddings to quantify how far input data deviate from the training distribution. This study introduces predictive variance as a novel indicator of FID reliability, revealing a clear link between embedding uncertainty and distributional shift. Empirical results demonstrate that this variance strongly correlates with the degree of out-of-distributionness on both an augmented ImageNet validation set and external medical imaging datasets, offering a new and robust perspective for evaluating synthetic medical image quality.

Current evaluations of generative models often report a single FID score, neglecting the randomness inherent in both training and sampling processes, which undermines reliability and reproducibility. This work is the first to model FID as a random variable jointly determined by training and sampling seeds. By training hundreds of SiT models on ImageNet at 256×256 resolution, we systematically quantify the sources of FID variance. Our analysis reveals that training seeds dominate FID variation—contributing approximately 3.2 times more than sampling seeds—and that optimal classifier-free guidance reduces FID dispersion by nearly half. Moreover, FID differences with a coefficient of variation (CoV) below 1.3% should be deemed statistically insignificant. Building on these insights, we propose a new evaluation paradigm incorporating multi-seed error bars and optimal guidance, substantially enhancing robustness and reproducibility in generative model assessment.

提出基于小波包变换的Fréchet Wavelet Distance(FWD)指标，解决现有生成图像评价指标对特定生成器和数据集的偏见问题，通过计算小波包系数空间的Fréchet距离，实现领域无关且更可解释的质量评估。

This work addresses the challenges of directly optimizing the Fréchet Distance (FD) in generative model training and the inconsistency between Fréchet Inception Distance (FID) and visual quality when relying solely on Inception features. The authors propose a differentiable optimization of FD in representation space, decoupling the large-scale samples required for FD estimation from minibatch-based gradient computation—enabling, for the first time, end-to-end training with FD as a loss function. Their approach compresses multi-step generators into high-performance single-step models without requiring distillation or adversarial training, and introduces FDr^k, a multi-representation-space evaluation metric. On ImageNet 256×256, the single-step generator achieves an FID of 0.72, markedly improving visual fidelity while revealing representation-dependent biases inherent in conventional FID evaluation.

This work addresses critical limitations of existing generative model evaluation metrics—such as the Fréchet Inception Distance (FID)—in sample complexity, computational efficiency, and robustness against adversarial attacks. The authors propose the Monge Inception Distance (MIND), which computes a one-dimensional optimal transport average of Inception features via sliced Wasserstein distance, thereby circumventing the need for high-dimensional mean and covariance estimation. MIND achieves evaluation performance comparable to FID with only 1k–5k samples, offering an order-of-magnitude improvement in sample efficiency and two orders of magnitude faster computation. Moreover, it demonstrates significantly enhanced robustness against adversarial manipulations such as moment matching. While maintaining high correlation with FID, MIND exhibits superior discriminative power, substantially facilitating efficient generative model development and iteration.

This study addresses the limitations of conventional image quality metrics—such as FID, SSIM, KID, IS, and LPIPS—in evaluating synthetic remote sensing imagery, demonstrating their poor alignment with both human perception and downstream task performance. By generating Earth observation images using deep generative models, the work systematically compares automatic metrics against human judgments and semantic segmentation accuracy for land cover classification. The findings reveal a significant misalignment: semantic-preserving perturbations substantially alter automatic scores without affecting human interpretability, and low-scoring synthetic samples can enhance segmentation performance when used in mixed training regimes. Moreover, metrics based on ImageNet feature spaces prove unreliable for geospatial data, underscoring the necessity of grounding synthetic data evaluation in task-specific performance and human assessment rather than generic image fidelity measures.

This work addresses a critical limitation of Training-Free Guidance (TFG) in high-noise regimes, where estimating a clean image from pure noise often causes the guidance signal to deviate from the data manifold, degrading generation quality. The study is the first to reveal that the choice of prediction target—ε, v, or x—profoundly influences TFG’s ability to preserve the data manifold. Theoretical analysis demonstrates that x-prediction directly yields the clean image, substantially reducing estimation error under high noise. To detect such manifold distortions overlooked by conventional metrics, the authors introduce a novel evaluation measure, guided-class FID (Child FID). Experiments on a newly curated fine-grained bird benchmark and style transfer tasks confirm that TFG with x-prediction significantly outperforms other prediction strategies in maintaining sample manifold consistency.

Existing generative models often suffer from mode collapse when optimizing the Fréchet Inception Distance (FID), struggling to balance sample quality and diversity. This work proposes a Recursive Tuning Mechanism (RTM), which, for the first time, integrates iterative latent refinement into the generative process, replacing the single-pass forward mapping used in architectures like StyleGAN2. RTM significantly enhances mode coverage without compromising fidelity. By combining implicit maximum likelihood estimation (IMLE) with a precision-recall evaluation framework, RTM achieves state-of-the-art precision and recall on CIFAR-10, CelebA-HQ, and nine few-shot benchmarks while maintaining excellent FID scores. Moreover, it substantially outperforms StyleGAN2 and its ADA variant on high-resolution tasks such as AFHQ-v1.