CodeEnhance: A Codebook-Driven Approach for Low-Light Image Enhancement

2024-04-08

arXiv.org

AI Summary

Low-light image enhancement (LLIE) faces two key challenges: ambiguity in restoration due to diverse illumination degradations, and trade-offs between noise suppression and brightness enhancement that compromise texture fidelity and color accuracy. To address these, we propose the first codebook-driven LLIE paradigm, formulating enhancement as a semantic-guided mapping from low-light inputs to high-quality discrete codebook representations. Our approach introduces three core innovations: (1) a Semantic Embedding Module (SEM) for hierarchical fusion of high-level semantics and low-level features; (2) a Customizable Codebook Shift (CS) mechanism enabling user-preference adaptation; and (3) an Interactive Feature Transformation (IFT) module for fine-grained texture and chromatic reconstruction. Integrating discrete vector quantization (VQ) with dynamic codebook adaptation, our method achieves state-of-the-art performance on both synthetic and real-world benchmarks—yielding substantial PSNR/SSIM gains—and demonstrates superior robustness to non-uniform illumination, noise, and color bias, along with enhanced visual quality and color fidelity.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Large Multimodal Models (LMMs)
• Search and Optimization: Learning to Search

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Large language models for search
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Low-light image enhancement (LLIE) aims to improve low-illumination images. However, existing methods face two challenges: (1) uncertainty in restoration from diverse brightness degradations; (2) loss of texture and color information caused by noise suppression and light enhancement. In this paper, we propose a novel enhancement approach, CodeEnhance, by leveraging quantized priors and image refinement to address these challenges. In particular, we reframe LLIE as learning an image-to-code mapping from low-light images to discrete codebook, which has been learned from high-quality images. To enhance this process, a Semantic Embedding Module (SEM) is introduced to integrate semantic information with low-level features, and a Codebook Shift (CS) mechanism, designed to adapt the pre-learned codebook to better suit the distinct characteristics of our low-light dataset. Additionally, we present an Interactive Feature Transformation (IFT) module to refine texture and color information during image reconstruction, allowing for interactive enhancement based on user preferences. Extensive experiments on both real-world and synthetic benchmarks demonstrate that the incorporation of prior knowledge and controllable information transfer significantly enhances LLIE performance in terms of quality and fidelity. The proposed CodeEnhance exhibits superior robustness to various degradations, including uneven illumination, noise, and color distortion.

Problem

• Addressing uncertainty in restoration from diverse brightness degradations
• Preventing loss of texture and color information during enhancement
• Adapting pre-learned codebook to low-light image characteristics

Innovation

• Leverages quantized priors for image enhancement
• Uses Semantic Embedding Module for feature integration
• Introduces Interactive Feature Transformation for refinement