UStyle: Waterbody Style Transfer of Underwater Scenes by Depth-Guided Feature Synthesis

2025-03-14

AI Summary

Underwater image style transfer is challenging due to severe geometric distortion and inconsistent stylization caused by wavelength- and depth-dependent attenuation, forward scattering, and backscattering artifacts—especially in unpaired settings. To address this, we propose the first reference-free underwater style transfer framework: (1) a physics-inspired, depth-aware whitening-and-color-transformation (DA-WCT) module grounded in underwater optical modeling; (2) UF7D, the first high-resolution benchmark dataset comprising seven distinct underwater styles; and (3) a multi-level constraint combining VGG- and CLIP-based semantic content preservation with a joint loss integrating color, luminance, structural, and frequency-domain fidelity. Extensive experiments demonstrate that our method significantly outperforms state-of-the-art approaches in style fidelity, structural integrity, and cross-water-type generalization. Both the code and UF7D dataset are publicly released.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Transfer, Domain Adaptation, Multi-Task Learning
• Reasoning under Uncertainty: Uncertainty Representations

Application Category

• Web Mining and Content Analysis: Large pretrained models with web data
• Responsible Web: Algorithmic accountability and transparency on the web
• Systems and Infrastructure for Web, Mobile and WoT: Experiences and lessons learnt from Web-based algorithms and system deployments

Abstract

The concept of waterbody style transfer remains largely unexplored in the underwater imaging and vision literature. Traditional image style transfer (STx) methods primarily focus on artistic and photorealistic blending, often failing to preserve object and scene geometry in images captured in high-scattering mediums such as underwater. The wavelength-dependent nonlinear attenuation and depth-dependent backscattering artifacts further complicate learning underwater image STx from unpaired data. This paper introduces UStyle, the first data-driven learning framework for transferring waterbody styles across underwater images without requiring prior reference images or scene information. We propose a novel depth-aware whitening and coloring transform (DA-WCT) mechanism that integrates physics-based waterbody synthesis to ensure perceptually consistent stylization while preserving scene structure. To enhance style transfer quality, we incorporate carefully designed loss functions that guide UStyle to maintain colorfulness, lightness, structural integrity, and frequency-domain characteristics, as well as high-level content in VGG and CLIP (contrastive language-image pretraining) feature spaces. By addressing domain-specific challenges, UStyle provides a robust framework for no-reference underwater image STx, surpassing state-of-the-art (SOTA) methods that rely solely on end-to-end reconstruction loss. Furthermore, we introduce the UF7D dataset, a curated collection of high-resolution underwater images spanning seven distinct waterbody styles, establishing a benchmark to support future research in underwater image STx. The UStyle inference pipeline and UF7D dataset are released at: https://github.com/uf-robopi/UStyle.

Problem

• Develops a framework for underwater image style transfer without reference images.
• Addresses challenges of wavelength and depth-dependent underwater image artifacts.
• Introduces a new dataset for benchmarking underwater image style transfer.

Innovation

• Depth-aware whitening and coloring transform (DA-WCT)
• Physics-based waterbody synthesis for consistent stylization
• Loss functions preserving color, structure, and frequency characteristics