🤖 AI Summary
This work addresses the challenge of generating thermal imagery from RGB inputs in aerial scenes, where paired RGB–thermal datasets are scarce. To this end, the authors propose a conditional U-Net architecture that incorporates weather condition metadata embedded into the bottleneck layer. The method further enhances image fidelity by integrating saturation and contrast adjustments as preprocessing steps and applying Gaussian blur as postprocessing within a Pix2Pix GAN framework. Systematic experiments demonstrate the critical role of auxiliary environmental information and tailored image processing in improving generation quality. Evaluated via five-fold cross-validation on a dataset of 612 image pairs, the proposed model significantly outperforms the ThermalGen baseline, achieving a PSNR of 14.55, an SSIM of 0.8095, and an LPIPS score as low as 0.1666.
📝 Abstract
Paired RGB-thermal data has shown significant utility across a range of applications, including image fusion, object tracking, and anomaly detection; however, its broader adoption is constrained by the limited availability of aligned RGB-thermal image pairs. RGB-to-thermal (and vice versa) image translation has emerged as a practical solution to this challenge. Prior approaches including conditional generative adversarial networks (cGANs) such as ThermalGAN and Scalable Interpolant Transformer (SiT)-based architectures such as ThermalGen have demonstrated strong potential for aerial-to-thermal image translation. In this work, we explore alternative architectures that prioritize simplicity while maintaining performance. Specifically, we propose a conditional U-Net that incorporates weather data at the bottleneck layer, complemented by targeted preprocessing and post-processing techniques applied within the Pix2Pix GAN architecture. We utilize a training set of 612 paired RGB and thermal images, and evaluate over 5-fold cross-validation, ultimately testing on a held-out test set. Our conditional U-Net model performed best, with a peak signal-to-noise ratio (PSNR) of 14.5485, structural similarity index measure (SSIM) of 0.8095, and learned perceptual image patch similarity (LPIPS) of 0.1666. These results outperformed the base ThermalGen model, which attained PSNR, SSIM, and LPIPS scores of 7.56, 0.2444, and 0.6317 respectively. We find that while saturation boost and contrast enhancement for preprocessing and Gaussian blur for post-processing provide observable improvements, the incorporation of conditioning data was most effective. Our findings cement the potential of integrating auxiliary metadata into thermal image generation, suggesting that such information can serve as a proxy for environmental conditions critical to accurate thermal reconstruction.