HyPER-GAN: Hybrid Patch-Based Image-to-Image Translation for Real-Time Photorealism Enhancement

2026-03-11

AI Summary

This work addresses the challenge that existing generative models often introduce visual artifacts and incur high computational costs when enhancing photorealism, hindering their deployment in real-time applications. To overcome this, the authors propose a lightweight U-Net-style generative adversarial network trained via a hybrid strategy that combines paired synthetic/augmented images with real-world image patches. This approach leverages a U-Net-based generator within an image-to-image translation framework to simultaneously preserve semantic consistency and significantly improve visual fidelity while reducing inference latency. Experimental results demonstrate that the proposed method outperforms state-of-the-art techniques in terms of inference speed, visual realism, and semantic robustness, thereby validating the effectiveness and practicality of the hybrid training strategy.

Technology Category

• Computer Vision: Generative Adversarial Networks (GANs) for Vision
• Machine Learning: Deep Generative Models & Autoencoders
• Natural Language Processing: Generation

Application Category

• Social Networks and Social Media: Generative AI / large language models and their impact on social systems
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Large language models for search

Abstract

Generative models are widely employed to enhance the photorealism of synthetic data for training computer vision algorithms. However, they often introduce visual artifacts that degrade the accuracy of these algorithms and require high computational resources, limiting their applicability in real-time training or evaluation scenarios. In this paper, we propose Hybrid Patch Enhanced Realism Generative Adversarial Network (HyPER-GAN), a lightweight image-to-image translation method based on a U-Net-style generator designed for real-time inference. The model is trained using paired synthetic and photorealism-enhanced images, complemented by a hybrid training strategy that incorporates matched patches from real-world data to improve visual realism and semantic consistency. Experimental results demonstrate that HyPER-GAN outperforms state-of-the-art paired image-to-image translation methods in terms of inference latency, visual realism, and semantic robustness. Moreover, it is illustrated that the proposed hybrid training strategy indeed improves visual quality and semantic consistency compared to training the model solely with paired synthetic and photorealism-enhanced images. Code and pretrained models are publicly available for download at: https://github.com/stefanos50/HyPER-GAN

Problem

• photorealism enhancement
• visual artifacts
• real-time inference
• image-to-image translation
• computational efficiency

Innovation

• HyPER-GAN
• real-time image translation
• hybrid patch training
• photorealism enhancement
• lightweight GAN